{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,27]],"date-time":"2025-12-27T09:31:11Z","timestamp":1766827871306,"version":"3.48.0"},"reference-count":34,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T00:00:00Z","timestamp":1760313600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T00:00:00Z","timestamp":1760313600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Health Care Manag Sci"],"published-print":{"date-parts":[[2025,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>In the context of diagnosis of COVID-19, this paper shows how to convert a Causal Network to a Large Language Model (LLM). The Causal Network was converted to the language model using prompts and completions. Prompts were composed from the full-factorial combination of the text associated with statistically significant variables in the Causal Network. Completions were based on the evaluation of the probability of COVID-19 using the Causal Network. The accuracy of the Causal Network and LLM was tested using two databases. The first database was based on a survey of 822 patients, collecting 12 direct (parents on the Markov blanket of COVID-19 diagnosis node), 7 indirect (associated with COVID-19 but not direct cause) symptoms of COVID-19. The second set was based on 80 patients reporting their symptoms in open-ended questions, often reporting some of the direct predictors and rarely reporting any indirect predictors of COVID-19. The accuracy of Causal Network and Markov blanket was tested using Area under the Receiver Operating Curve (AUROC). When indirect information was available, the Causal Network model (AUROC\u2009=\u20090.91) was significantly more accurate than the LLM (AUROC\u2009=\u20090.88), even though LLM model was trained to duplicate predictions of the Causal Network. Where the indirect information was not available, both models had lower accuracy (AUROC of 0.75 and 0.76). The accuracy of LLM depends not only on patterns among direct predictors of the outcome but also data not reported to the LLM. Conversational LLMs need to go beyond information the patient supplies and proactively ask about missing, typically indirect, information.<\/jats:p>","DOI":"10.1007\/s10729-025-09724-8","type":"journal-article","created":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T07:47:42Z","timestamp":1760341662000},"page":"575-582","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Causal networks guiding large language models: application to COVID-19"],"prefix":"10.1007","volume":"28","author":[{"given":"Farrokh","family":"Alemi","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kevin James","family":"Lybarger","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jee","family":"Vang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-6771-5911","authenticated-orcid":false,"given":"Yili","family":"Lin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hadeel R. A.","family":"Elyazori","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vladimir Franzuela","family":"Cardenas","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,10,13]]},"reference":[{"key":"9724_CR1","doi-asserted-by":"crossref","unstructured":"B\u00fcy\u00fcktahtak\u0131n \u0130E, des Bordes E, K\u0131b\u0131\u015f EY (2018) A new epidemics\u2013logistics model: Insights into controlling the Ebola virus disease in West Africa. European Journal of Operational Research, Elsevier, 265(3):1046\u20131063","DOI":"10.1016\/j.ejor.2017.08.037"},{"issue":"6","key":"9724_CR2","doi-asserted-by":"publisher","first-page":"855","DOI":"10.1038\/s41591-020-0883-7","volume":"26","author":"G Giordano","year":"2020","unstructured":"Giordano G, Blanchini F, Bruno R, Colaneri P, Di Filippo A, Di Matteo A, Colaneri M (2020) Modelling the COVID-19 epidemic and implementation of population-wide interventions in Italy. Nat Med 26(6):855\u2013860","journal-title":"Nat Med"},{"issue":"4","key":"9724_CR3","doi-asserted-by":"publisher","first-page":"1737","DOI":"10.1109\/JBHI.2021.3123192","volume":"26","author":"X Luo","year":"2022","unstructured":"Luo X, Gandhi P, Storey S, Huang K (2022) A deep language model for symptom extraction from clinical text and its application to extract COVID-19 symptoms from social media. IEEE J Biomed Health Inform 26(4):1737\u20131748. https:\/\/doi.org\/10.1109\/JBHI.2021.3123192","journal-title":"IEEE J Biomed Health Inform"},{"key":"9724_CR4","doi-asserted-by":"publisher","DOI":"10.1155\/2022\/1306664","author":"D Sinwar","year":"2022","unstructured":"Sinwar D, Dhaka VS, Tesfaye BA, Raghuwanshi G, Kumar A, Maakar SK, Agrawal S (2022) Artificial intelligence and deep learning assisted rapid diagnosis of COVID-19 from chest radiographical images: a survey. Contrast Media Mol Imaging. https:\/\/doi.org\/10.1155\/2022\/1306664","journal-title":"Contrast Media Mol Imaging"},{"issue":"1","key":"9724_CR5","doi-asserted-by":"publisher","DOI":"10.3390\/v14010063","volume":"14","author":"JM Anto\u00f1anzas","year":"2021","unstructured":"Anto\u00f1anzas JM, Perramon A, L\u00f3pez C, Boneta M, Aguilera C, Capdevila R, Gatell A, Serrano P, Poblet M, Canadell D, Vil\u00e0 M, Catas\u00fas G, Valldep\u00e9rez C, Catal\u00e0 M, Soler-Palac\u00edn P, Prats C, Soriano-Arandes A, The Copedi-Cat Research Group (2021) Symptom-based predictive model of COVID-19 disease in children. Viruses 14(1):63. https:\/\/doi.org\/10.3390\/v14010063","journal-title":"Viruses"},{"key":"9724_CR6","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2024.109211","volume":"182","author":"B Fakieh","year":"2024","unstructured":"Fakieh B, Saleem F (2024) COVID-19 from symptoms to prediction: a statistical and machine learning approach. Comput Biol Med 182:109211","journal-title":"Comput Biol Med"},{"issue":"7","key":"9724_CR7","doi-asserted-by":"publisher","first-page":"832","DOI":"10.1002\/alr.22602","volume":"10","author":"LT Roland","year":"2020","unstructured":"Roland LT, Gurrola JG 2nd, Loftus PA, Cheung SW, Chang JL (2020) Smell and taste symptom-based predictive model for COVID-19 diagnosis. Int Forum Allergy Rhinol 10(7):832\u2013838. https:\/\/doi.org\/10.1002\/alr.22602","journal-title":"Int Forum Allergy Rhinol"},{"issue":"Suppl 1","key":"9724_CR8","doi-asserted-by":"publisher","first-page":"S11","DOI":"10.1097\/QMH.0000000000000404","volume":"32","author":"F Alemi","year":"2023","unstructured":"Alemi F, Vang J, Bagais WH, Guralnik E, Wojtusiak J, Moeller FG, Schilling J, Peterson R, Roess A, Jain P (2023) Combined symptom screening and at-home tests for COVID-19. Qual Manag Health Care. 32(Suppl 1):S11-20. https:\/\/doi.org\/10.1097\/QMH.0000000000000404","journal-title":"Qual Manag Health Care."},{"issue":"Suppl 1","key":"9724_CR9","doi-asserted-by":"publisher","first-page":"S3","DOI":"10.1097\/QMH.0000000000000398","volume":"32","author":"F Alemi","year":"2023","unstructured":"Alemi F, Guralnik E, Vang J, Wojtusiak J, Peterson R, Roess A, Jain P (2023) Guidelines for triage of COVID-19 patients presenting with multisystemic symptoms. Qual Manag Health Care 32(Suppl 1):S3\u2013S10. https:\/\/doi.org\/10.1097\/QMH.0000000000000398","journal-title":"Qual Manag Health Care"},{"issue":"Suppl 1","key":"9724_CR10","doi-asserted-by":"publisher","first-page":"S21","DOI":"10.1097\/QMH.0000000000000399","volume":"32","author":"J Wojtusiak","year":"2023","unstructured":"Wojtusiak J, Bagais W, Vang J, Guralnik E, Roess A, Alemi F (2023) The role of symptom clusters in triage of COVID-19 patients. Qual Manag Health Care 32(Suppl 1):S21\u2013S28. https:\/\/doi.org\/10.1097\/QMH.0000000000000399","journal-title":"Qual Manag Health Care"},{"issue":"2","key":"9724_CR11","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1097\/QMH.0000000000000339","volume":"31","author":"F Alemi","year":"2022","unstructured":"Alemi F, Vang J, Guralnik E, Roess A (2022) Modeling the probability of COVID-19 based on symptom screening and prevalence of influenza and influenza-like illnesses. Qual Manag Health Care 31(2):85\u201391. https:\/\/doi.org\/10.1097\/QMH.0000000000000339","journal-title":"Qual Manag Health Care"},{"issue":"Suppl 1","key":"9724_CR12","doi-asserted-by":"publisher","first-page":"S29","DOI":"10.1097\/QMH.0000000000000397","volume":"32","author":"J Wojtusiak","year":"2023","unstructured":"Wojtusiak J, Bagais W, Vang J, Roess A, Alemi F (2023) Order of occurrence of COVID-19 symptoms. Qual Manag Health Care 32(Suppl 1):S29\u2013S34. https:\/\/doi.org\/10.1097\/QMH.0000000000000397","journal-title":"Qual Manag Health Care"},{"issue":"3","key":"9724_CR13","doi-asserted-by":"publisher","DOI":"10.3390\/healthcare10030541","volume":"10","author":"MM Ahsan","year":"2022","unstructured":"Ahsan MM, Luna SA, Siddique Z (2022) Machine-learning-based disease diagnosis: a comprehensive review. Healthcare 10(3):541. https:\/\/doi.org\/10.3390\/healthcare10030541","journal-title":"Healthcare"},{"key":"9724_CR14","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1146\/annurev.bioeng.8.061505.095802","volume":"8","author":"P Sajda","year":"2006","unstructured":"Sajda P (2006) Machine learning for detection and diagnosis of disease. Annu Rev Biomed Eng 8:537\u2013565. https:\/\/doi.org\/10.1146\/annurev.bioeng.8.061505.095802","journal-title":"Annu Rev Biomed Eng"},{"key":"9724_CR15","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2023.104429","volume":"143","author":"P Roman-Naranjo","year":"2023","unstructured":"Roman-Naranjo P, Parra-Perez AM, Lopez-Escamez JA (2023) A systematic review on machine learning approaches in the diagnosis and prognosis of rare genetic diseases. J Biomed Inform 143:104429. https:\/\/doi.org\/10.1016\/j.jbi.2023.104429","journal-title":"J Biomed Inform"},{"issue":"5","key":"9724_CR16","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1093\/bib\/bbt020","volume":"14","author":"D de Ridder","year":"2013","unstructured":"de Ridder D, de Ridder J, Reinders MJ (2013) Pattern recognition in bioinformatics. Brief Bioinform 14(5):633\u2013647. https:\/\/doi.org\/10.1093\/bib\/bbt020","journal-title":"Brief Bioinform"},{"issue":"5699","key":"9724_CR17","doi-asserted-by":"publisher","first-page":"1194","DOI":"10.1126\/science.1100709","volume":"306","author":"ZE Perlman","year":"2004","unstructured":"Perlman ZE, Slack MD, Feng Y, Mitchison TJ, Wu LF, Altschuler SJ (2004) Multidimensional drug profiling by automated microscopy. Science 306(5699):1194\u20131198. https:\/\/doi.org\/10.1126\/science.1100709","journal-title":"Science"},{"issue":"1","key":"9724_CR18","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1038\/nm.4246","volume":"23","author":"AT Drysdale","year":"2017","unstructured":"Drysdale AT, Grosenick L, Downar J, Dunlop K, Mansouri F, Meng Y, Fetcho RN, Zebley B, Oathes DJ, Etkin A, Schatzberg AF, Sudheimer K, Keller J, Mayberg HS, Gunning FM, Alexopoulos GS, Fox MD, Pascual-Leone A, Voss HU, Casey BJ, Dubin MJ, Liston C (2017) Resting-state connectivity biomarkers define neurophysiological subtypes of depression. Nat Med 23(1):28\u201338. https:\/\/doi.org\/10.1038\/nm.4246","journal-title":"Nat Med"},{"issue":"181","key":"9724_CR19","doi-asserted-by":"publisher","first-page":"181ra50","DOI":"10.1126\/scitranslmed.3005974","volume":"5","author":"WY Cheng","year":"2013","unstructured":"Cheng WY, Ou Yang TH, Anastassiou D (2013) Development of a prognostic model for breast cancer survival in an open challenge environment. Sci Transl Med 5(181):181ra50. https:\/\/doi.org\/10.1126\/scitranslmed.3005974","journal-title":"Sci Transl Med"},{"issue":"2","key":"9724_CR20","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1016\/j.jphs.2017.01.003","volume":"133","author":"M Gao","year":"2017","unstructured":"Gao M, Igata H, Takeuchi A, Sato K, Ikegaya Y (2017) Machine learning-based prediction of adverse drug effects: an example of seizure-inducing compounds. J Pharmacol Sci 133(2):70\u201378. https:\/\/doi.org\/10.1016\/j.jphs.2017.01.003","journal-title":"J Pharmacol Sci"},{"issue":"1","key":"9724_CR21","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-020-00372-6","volume":"4","author":"Y Zoabi","year":"2021","unstructured":"Zoabi Y, Deri-Rozov S, Shomron N (2021) Machine learning-based prediction of COVID-19 diagnosis based on symptoms. NPJ Digit Med 4(1):3. https:\/\/doi.org\/10.1038\/s41746-020-00372-6","journal-title":"NPJ Digit Med"},{"issue":"9","key":"9724_CR22","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pgen.1009811","volume":"17","author":"R Howey","year":"2021","unstructured":"Howey R et al (2021) A bayesian network approach incorporating imputation of missing data enables exploratory analysis of complex causal biological relationships. PLoS Genet 17(9):e1009811","journal-title":"PLoS Genet"},{"issue":"4","key":"9724_CR23","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1097\/QMH.0000000000000272","volume":"29","author":"F Alemi","year":"2020","unstructured":"Alemi F (2020) Constructing causal networks through regressions: a tutorial. Qual Manag Health Care 29(4):270\u2013278. https:\/\/doi.org\/10.1097\/QMH.0000000000000272","journal-title":"Qual Manag Health Care"},{"issue":"18","key":"9724_CR24","doi-asserted-by":"publisher","first-page":"i517","DOI":"10.1093\/bioinformatics\/btq377","volume":"26","author":"A Shojaie","year":"2010","unstructured":"Shojaie A, Michailidis G (2010) Discovering graphical granger causality using the truncating lasso penalty. Bioinformatics 26(18):i517\u2013i523. https:\/\/doi.org\/10.1093\/bioinformatics\/btq377","journal-title":"Bioinformatics"},{"key":"9724_CR25","unstructured":"Who's #1? | Princeton University Press. Available from: https:\/\/press.princeton.edu\/books\/hardcover\/9780691154220\/whos-1"},{"issue":"3","key":"9724_CR26","doi-asserted-by":"publisher","first-page":"544","DOI":"10.1136\/amiajnl-2012-000929","volume":"20","author":"DM Maslove","year":"2013","unstructured":"Maslove DM, Podchiyska T, Lowe HJ (2013) Discretization of continuous features in clinical datasets. J Am Med Inform Assoc 20(3):544\u2013553","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"9724_CR27","first-page":"18","volume":"93","author":"MJ Hashim","year":"2024","unstructured":"Hashim MJ (2024) Verbal probability terms for communicating clinical risk - a systematic review. Ulster Med J. 93(1):18\u201323","journal-title":"Ulster Med J."},{"key":"9724_CR28","doi-asserted-by":"publisher","unstructured":"Lu K, Zhang S, Chen X (2019) Goal-oriented dialogue policy learning from failures. arXiv [preprint]. https:\/\/doi.org\/10.48550\/arXiv.1808.06497","DOI":"10.48550\/arXiv.1808.06497"},{"key":"9724_CR29","doi-asserted-by":"publisher","unstructured":"Su P, Gasic M, Mrksic N, RojasBarahona L, Ultes S, Vandyke D, Wen T, Young S Continuously learning neural dialogue management. arXiv [preprint]. https:\/\/doi.org\/10.48550\/arXiv.2307.09021","DOI":"10.48550\/arXiv.2307.09021"},{"key":"9724_CR30","first-page":"665","volume":"1","author":"JD Williams","year":"2017","unstructured":"Williams JD, Atui KA, Zweig G (2017) Hybrid code networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning. Proc Assoc Comput Linguist 1:665\u2013677","journal-title":"Proc Assoc Comput Linguist"},{"key":"9724_CR31","doi-asserted-by":"publisher","first-page":"2060","DOI":"10.18653\/v1\/N18-1187","volume":"1","author":"B Liu","year":"2018","unstructured":"Liu B, T\u00fcr G, Hakkani-Tur D, Shah P, Heck L (2018) Dialogue learning with human teaching and feedback in end-to-end trainable task-oriented dialogue systems. In Proc NAACL-HLT 1:2060\u20132069. https:\/\/doi.org\/10.18653\/v1\/N18-1187","journal-title":"In Proc NAACL-HLT"},{"key":"9724_CR32","first-page":"482","volume":"2017","author":"B Liu","year":"2017","unstructured":"Liu B, Lane I (2017) Iterative policy learning in end-to-end trainable task-oriented neural dialog models. In Proc IEEE Workshop Autom Speech Recognit Underst 2017:482\u2013489","journal-title":"In Proc IEEE Workshop Autom Speech Recognit Underst"},{"key":"9724_CR33","doi-asserted-by":"publisher","unstructured":"Takanobu R, Zhu H, Huang M. Guided dialog policy learning: reward estimation for multi-domain task-oriented dialog. In: Proc Conf Empir Methods Nat Lang Process. 2019 Nov:100\u2013110. https:\/\/doi.org\/10.18653\/v1\/D19-1010","DOI":"10.18653\/v1\/D19-1010"},{"issue":"3","key":"9724_CR34","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1010967","volume":"19","author":"Y Wu","year":"2023","unstructured":"Wu Y, Mascaro S, Bhuiyan M, Fathima P, Mace AO, Nicol MP, Richmond PC, Kirkham LA, Dymock M, Foley DA, McLeod C, Borland ML, Martin A, Williams PCM, Marsh JA, Snelling TL, Blyth CC (2023) Predicting the causative pathogen among children with pneumonia using a causal bayesian network. PLoS Comput Biol 19(3):e1010967. https:\/\/doi.org\/10.1371\/journal.pcbi.1010967","journal-title":"PLoS Comput Biol"}],"container-title":["Health Care Management Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10729-025-09724-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10729-025-09724-8","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10729-025-09724-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,27]],"date-time":"2025-12-27T09:27:24Z","timestamp":1766827644000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10729-025-09724-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,13]]},"references-count":34,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,12]]}},"alternative-id":["9724"],"URL":"https:\/\/doi.org\/10.1007\/s10729-025-09724-8","relation":{},"ISSN":["1386-9620","1572-9389"],"issn-type":[{"type":"print","value":"1386-9620"},{"type":"electronic","value":"1572-9389"}],"subject":[],"published":{"date-parts":[[2025,10,13]]},"assertion":[{"value":"21 August 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 August 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 October 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The author has no affiliations with or involvement in any organization or entity with any financial interest in the subject matter or materials discussed in this manuscript. The project has received comprehensive institutional approval, ensuring adherence to ethical standards and regulatory compliance. Specifically:I. Western Institutional Review Board (IRB) Approval:- Study Number: 1309332- The project was meticulously reviewed and approved by the Western IRB, affirming that all protocols meet stringent ethical guidelines for human subjects research.II. Virginia Commonwealth University (VCU) Authorization:- Study Number: HM20022035- The data collection effort was sanctioned by VCU, which deferred to the Western IRB for oversight. This deferment underscores VCU's commitment to collaborative compliance with established ethical standards.III. George Mason University (GMU) IRB Approval:- Study Number: 1743684\u20131- The analysis of de-identified data received authorization from the GMU IRB, ensuring that all research activities adhere to the highest ethical standards for data confidentiality and participant privacy.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}