{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T23:25:31Z","timestamp":1782429931522,"version":"3.54.5"},"reference-count":186,"publisher":"IOP Publishing","issue":"1","license":[{"start":{"date-parts":[[2025,1,31]],"date-time":"2025-01-31T00:00:00Z","timestamp":1738281600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,1,31]],"date-time":"2025-01-31T00:00:00Z","timestamp":1738281600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"name":"European Research Council","award":["ERC Synergy Grant USMILE (grant agreement 855187)"],"award-info":[{"award-number":["ERC Synergy Grant USMILE (grant agreement 855187)"]}]},{"name":"Fundaci\u00f3n BBVA","award":["Scale"],"award-info":[{"award-number":["Scale"]}]},{"DOI":"10.13039\/501100016386","name":"Conselleria de Innovaci\u00f3n, Universidades, Ciencia y Sociedad Digital, Generalitat Valenciana","doi-asserted-by":"crossref","award":["Prometeo (CIPROM\/2021\/056)"],"award-info":[{"award-number":["Prometeo (CIPROM\/2021\/056)"]}],"id":[{"id":"10.13039\/501100016386","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100006112","name":"Microsoft Research","doi-asserted-by":"crossref","award":["Causal4Africa"],"award-info":[{"award-number":["Causal4Africa"]}],"id":[{"id":"10.13039\/100006112","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2025,3,31]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Towards the goal of understanding the causal structure underlying complex systems\u2014such as the Earth, the climate, or the brain\u2014integrating Large language models (LLMs) with data-driven and domain-expertise-driven approaches has the potential to become a game-changer, especially in data and expertise-limited scenarios. Debates persist around LLMs\u2019 causal reasoning capacities. However, rather than engaging in philosophical debates, we propose integrating LLMs into a scientific framework for causal hypothesis generation alongside expert knowledge and data. Our goals include formalizing LLMs as probabilistic imperfect experts, developing adaptive methods for causal hypothesis generation, and establishing universal benchmarks for comprehensive comparisons. Specifically, we introduce a spectrum of integration methods for experts, LLMs, and data-driven approaches. We review existing approaches for causal hypothesis generation and classify them within this spectrum. As an example, our hybrid (LLM + data) causal discovery algorithm illustrates ways for deeper integration. Characterizing imperfect experts along dimensions such as (1) reliability, (2) consistency, (3) uncertainty, and (4) content vs. reasoning are emphasized for developing adaptable methods. Lastly, we stress the importance of model-agnostic benchmarks.<\/jats:p>","DOI":"10.1088\/2632-2153\/ada47f","type":"journal-article","created":{"date-parts":[[2024,12,31]],"date-time":"2024-12-31T17:54:56Z","timestamp":1735667696000},"page":"013001","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Large language models for causal hypothesis generation in science"],"prefix":"10.1088","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2286-7487","authenticated-orcid":true,"given":"Kai-Hendrik","family":"Cohrs","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8410-6635","authenticated-orcid":false,"given":"Emiliano","family":"Diaz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5506-2872","authenticated-orcid":true,"given":"Vasileios","family":"Sitokonstantinou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6708-1103","authenticated-orcid":true,"given":"Gherardo","family":"Varando","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1683-2138","authenticated-orcid":false,"given":"Gustau","family":"Camps-Valls","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2025,1,31]]},"reference":[{"key":"mlstada47fbib1","article-title":"Living guidelines on the responsible use of generative AI in research","author":"European Commission, Directorate-General for Research and Innovation","year":"2024"},{"key":"mlstada47fbib2","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2024.naacl-long.219","article-title":"Can knowledge graphs reduce hallucinations in LLMs?: A survey","author":"Agrawal","year":"2024"},{"key":"mlstada47fbib3","doi-asserted-by":"publisher","first-page":"2154","DOI":"10.1109\/TPAMI.2016.2636828","article-title":"Exploiting experts\u2019 knowledge for structure learning of Bayesian networks","volume":"39","author":"Amirkhani","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"mlstada47fbib4","doi-asserted-by":"crossref","DOI":"10.1609\/aaaiss.v4i1.31764","article-title":"Cause and effect: can large language models truly understand causality?","author":"Ashwani","year":"2024"},{"key":"mlstada47fbib5","article-title":"Causal structure learning supervised by large language model","author":"Ban","year":"2023"},{"key":"mlstada47fbib6","article-title":"LLMs with chain-of-thought are non-causal reasoners","author":"Bao","year":"2024"},{"key":"mlstada47fbib7","doi-asserted-by":"publisher","first-page":"pp 507","DOI":"10.1145\/3501714.3501743","article-title":"On pearl\u2019s hierarchy and the foundations of causal inference","author":"Bareinboim","year":"2022"},{"key":"mlstada47fbib8","first-page":"pp 17682","article-title":"Graph of thoughts: solving elaborate problems with large language models","volume":"vol 38","author":"Besta","year":"2024"},{"key":"mlstada47fbib9","first-page":"pp 3098","article-title":"ChatGPT is a knowledgeable but inexperienced solver: an investigation of commonsense problem in large language models","author":"Bian","year":"2024"},{"key":"mlstada47fbib10","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1038\/s42254-023-00581-4","article-title":"Science in the age of large language models","volume":"5","author":"Birhane","year":"2023","journal-title":"Nat. Rev. Phys."},{"key":"mlstada47fbib11","doi-asserted-by":"publisher","first-page":"693","DOI":"10.1038\/d41586-023-03266-1","article-title":"Living guidelines for generative AI\u2013why scientists must oversee its use","volume":"622","author":"Bockting","year":"2023","journal-title":"Nature"},{"key":"mlstada47fbib12","first-page":"pp 3296","article-title":"CausalQA: a benchmark for causal question answering","author":"Bondarenko","year":"2022"},{"key":"mlstada47fbib13","doi-asserted-by":"publisher","first-page":"2885","DOI":"10.1214\/21-AOS2064","article-title":"Foundations of structural causal models with cycles and latent variables","volume":"49","author":"Bongers","year":"2021","journal-title":"Ann. Stat."},{"key":"mlstada47fbib14","first-page":"pp 427","article-title":"Incorporating causal prior knowledge as path-constraints in Bayesian networks and maximal ancestral graphs","author":"Borboudakis","year":"2012"},{"key":"mlstada47fbib15","first-page":"pp 102","article-title":"Scoring and searching over Bayesian networks with causal and associative priors","author":"Borboudakis","year":"2013"},{"key":"mlstada47fbib16","article-title":"Causality and reasoning: the Monty Hall dilemma","author":"Burns","year":"2003"},{"key":"mlstada47fbib17","article-title":"Is knowledge all large language models needed for causal reasoning?","author":"Cai","year":"2024"},{"key":"mlstada47fbib18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.physrep.2023.10.005","article-title":"Discovering causal relations and equations from data","volume":"1044","author":"Camps-Valls","year":"2023","journal-title":"Phys. Rep."},{"key":"mlstada47fbib19","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1016\/j.asoc.2018.03.021","article-title":"Physics-aware Gaussian processes in remote sensing","volume":"68","author":"Camps-Valls","year":"2018","journal-title":"Appl. Soft Comput."},{"key":"mlstada47fbib20","first-page":"pp 77","article-title":"The event StoryLine corpus: a new benchmark for causal and temporal relation extraction","author":"Caselli","year":"2017"},{"key":"mlstada47fbib21","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1016\/S0888-613X(99)00041-9","article-title":"Priors on network structures. Biasing the search for Bayesian networks","volume":"24","author":"Castelo","year":"2000","journal-title":"Int. J. Approx. Reason."},{"key":"mlstada47fbib22","article-title":"Evaluating the impact of humanitarian aid on food security","author":"Cerd\u00e0-Bautista","year":"2023"},{"key":"mlstada47fbib23","first-page":"2333","article-title":"Learning Bayesian networks with ancestral constraints","volume":"vol 29","author":"Chen","year":"2016"},{"key":"mlstada47fbib24","first-page":"pp 12881","article-title":"Probing the \u201ccreativity\u201d of large language models: can models produce divergent semantic association?","author":"Chen","year":"2023"},{"key":"mlstada47fbib25","article-title":"Mitigating prior errors in causal structure learning: towards LLM driven prior knowledge","author":"Chen","year":"2023"},{"key":"mlstada47fbib26","article-title":"Boosting of thoughts: trial-and-error problem solving with large language models","author":"Chen","year":"2024"},{"key":"mlstada47fbib27","doi-asserted-by":"publisher","first-page":"C1","DOI":"10.1111\/ectj.12097","article-title":"Double\/debiased machine learning for treatment and structural parameters","volume":"21","author":"Chernozhukov","year":"2018","journal-title":"J. Econom."},{"key":"mlstada47fbib28","first-page":"507","article-title":"Optimal structure identification with greedy search","volume":"3","author":"Chickering","year":"2002","journal-title":"J. Mach. Learn. Res."},{"key":"mlstada47fbib29","first-page":"445","article-title":"Learning equivalence classes of Bayesian-network structures","volume":"2","author":"Chickering","year":"2002","journal-title":"J. Mach. Learn. Res."},{"key":"mlstada47fbib30","first-page":"pp 443","article-title":"Greedy equivalence search in the presence of latent confounders","volume":"vol 180","author":"Claassen","year":"2022"},{"key":"mlstada47fbib31","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1088\/2632-2153\/ad5a60","article-title":"Causal hybrid modeling with double machine learning\u2014applications in carbon flux modeling","volume":"5","author":"Cohrs","year":"2024","journal-title":"Mach. Learn.: Sci. Technol"},{"key":"mlstada47fbib32","article-title":"Large language models for constrained-based causal discovery","author":"Cohrs","year":"2024"},{"key":"mlstada47fbib33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10115-023-01858-x","article-title":"The impact of prior knowledge on causal structure learning","volume":"65","author":"Constantinou","year":"2023","journal-title":"Knowl. Inf. Syst."},{"key":"mlstada47fbib34","doi-asserted-by":"publisher","DOI":"10.1088\/1748-9326\/ac6762","article-title":"Physics-aware nonparametric regression models for earth data analysis","volume":"17","author":"Cort\u00e9s-Andr\u00e9s","year":"2022","journal-title":"Environ. Res. Lett."},{"key":"mlstada47fbib35","doi-asserted-by":"publisher","DOI":"10.1016\/j.gloepi.2023.100102","article-title":"Causal reasoning about epidemiological associations in conversational AI","volume":"5","author":"Cox","year":"2023","journal-title":"Glob. Epidemiol."},{"key":"mlstada47fbib36","article-title":"Human-in-the-loop causal discovery under latent confounding using ancestral GFlowNets","author":"da Silva","year":"2023"},{"key":"mlstada47fbib37","article-title":"Large language models are effective priors for causal graph discovery","author":"Darvariu","year":"2024"},{"key":"mlstada47fbib38","first-page":"663","article-title":"Efficient structure learning of Bayesian networks using constraints","volume":"12","author":"De Campos","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"mlstada47fbib39","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1016\/j.ijar.2006.06.009","article-title":"Bayesian network learning algorithms using structural restrictions","volume":"45","author":"de Campos","year":"2007","journal-title":"Int. J. Approx. Reason."},{"key":"mlstada47fbib40","first-page":"pp 6491","article-title":"Editing factual knowledge in language models","author":"De Cao","year":"2021"},{"key":"mlstada47fbib41","doi-asserted-by":"publisher","first-page":"pp 248","DOI":"10.1109\/CVPR.2009.5206848","article-title":"ImageNet: a large-scale hierarchical image database","author":"Deng","year":"2009"},{"key":"mlstada47fbib42","doi-asserted-by":"publisher","first-page":"252","DOI":"10.1016\/j.cels.2020.08.003","article-title":"SERGIO: a single-cell expression simulator guided by gene regulatory networks","volume":"11","author":"Dibaeinia","year":"2020","journal-title":"Cell Syst."},{"key":"mlstada47fbib43","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1111\/j.1467-9868.2007.00634.x","article-title":"Graphical models for marked point processes based on local independence","volume":"70","author":"Didelez","year":"2008","journal-title":"J. R. Stat. Soc. B"},{"key":"mlstada47fbib44","doi-asserted-by":"publisher","first-page":"602","DOI":"10.1016\/j.ijar.2016.12.012","article-title":"An empirical comparison of popular structure learning algorithms with a view to gene network inference","volume":"88","author":"Djordjilovi\u0107","year":"2017","journal-title":"Int. J. Approx. Reason."},{"key":"mlstada47fbib45","first-page":"pp 432","article-title":"e-CARE: a new dataset for exploring explainable causal reasoning","author":"Du","year":"2022"},{"key":"mlstada47fbib46","first-page":"pp 57","article-title":"Causal thinking in science: how scientists and students interpret the unexpected","author":"Dunbar","year":"2004"},{"key":"mlstada47fbib47","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2023.110694","article-title":"On causal structural learning algorithms: Oracles\u2019 simulations and considerations","volume":"276","author":"Farnia","year":"2023","journal-title":"Knowl.-Based Syst."},{"key":"mlstada47fbib48","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1162\/coli_a_00404","article-title":"CausaLM: causal model explanation through counterfactual language models","volume":"47","author":"Feder","year":"2021","journal-title":"Comput. Linguist."},{"key":"mlstada47fbib49","first-page":"3133","article-title":"Do we need hundreds of classifiers to solve real world classification problems?","volume":"15","author":"Fern\u00e1ndez-Delgado","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"mlstada47fbib50","article-title":"On the creativity of large language models","author":"Franceschelli","year":"2023"},{"key":"mlstada47fbib51","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1023\/A:1020249912095","article-title":"Being Bayesian about network structure. A Bayesian approach to structure discovery in Bayesian networks","volume":"50","author":"Friedman","year":"2003","journal-title":"Mach. Learn."},{"key":"mlstada47fbib52","doi-asserted-by":"publisher","first-page":"1273","DOI":"10.1016\/S1053-8119(03)00202-7","article-title":"Dynamic causal modelling","volume":"19","author":"Friston","year":"2003","journal-title":"NeuroImage"},{"key":"mlstada47fbib53","first-page":"pp 2126","article-title":"CRASS: a novel data set and benchmark to test counterfactual reasoning of large language models","author":"Frohberg","year":"2022"},{"key":"mlstada47fbib54","article-title":"LLM hallucination index","author":"Galileo","year":"2023"},{"key":"mlstada47fbib55","first-page":"pp 11111","article-title":"Is ChatGPT a good causal reasoner? A comprehensive evaluation","author":"Gao","year":"2023"},{"key":"mlstada47fbib56","article-title":"Faithful explanations of black-box NLP models using LLM-generated counterfactuals","author":"Gat","year":"2023"},{"key":"mlstada47fbib57","doi-asserted-by":"publisher","first-page":"pp 6577","DOI":"10.18653\/v1\/2024.naacl-long.366","article-title":"A survey of confidence estimation and calibration in large language models","author":"Geng","year":"2024"},{"key":"mlstada47fbib58","first-page":"pp 12615","article-title":"High-recall causal discovery for autocorrelated time series with latent confounders","volume":"vol 33","author":"Gerhardus","year":"2020"},{"key":"mlstada47fbib59","doi-asserted-by":"publisher","first-page":"214","DOI":"10.5210\/ojphi.v5i2.4623","article-title":"Google scholar is not enough to be used alone for systematic reviews","volume":"5","author":"Giustini","year":"2013","journal-title":"Online J. Public Health Inform."},{"key":"mlstada47fbib60","doi-asserted-by":"publisher","first-page":"524","DOI":"10.3389\/fgene.2019.00524","article-title":"Review of causal discovery methods based on graphical models","volume":"10","author":"Glymour","year":"2019","journal-title":"Front. Genet."},{"key":"mlstada47fbib61","article-title":"Connecting large language models with evolutionary algorithms yields powerful prompt optimizers","author":"Guo","year":"2024"},{"key":"mlstada47fbib62","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0138237","article-title":"The role of google scholar in evidence reviews and its applicability to grey literature searching","volume":"10","author":"Haddaway","year":"2015","journal-title":"PLoS One"},{"key":"mlstada47fbib63","first-page":"pp 274","article-title":"Learning Bayesian networks: a unification for discrete and Gaussian domains","author":"Heckerman","year":"1995"},{"key":"mlstada47fbib64","article-title":"Adjustment identification distance: a gadjid for causal structure learning","author":"Henckel","year":"2024"},{"key":"mlstada47fbib65","article-title":"WikiWhy: answering and explaining cause-and-effect questions","author":"Ho","year":"2023"},{"key":"mlstada47fbib66","article-title":"Investigating causal understanding in LLMs","author":"Hobbhahn","year":"2022"},{"key":"mlstada47fbib67","article-title":"Towards understanding factual knowledge of large language models","author":"Hu","year":"2024"},{"key":"mlstada47fbib68","article-title":"Uncertainty of thoughts: uncertainty-aware planning enhances information seeking in large language models","author":"Hu","year":"2024"},{"key":"mlstada47fbib69","article-title":"A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions","author":"Huang","year":"2023"},{"key":"mlstada47fbib70","doi-asserted-by":"publisher","DOI":"10.3389\/fdata.2021.642182","article-title":"Benchmarking of data-driven causality discovery approaches in the interactions of Arctic Sea ice and atmosphere","volume":"4","author":"Huang","year":"2021","journal-title":"Front. Big Data"},{"key":"mlstada47fbib71","doi-asserted-by":"publisher","first-page":"208","DOI":"10.1108\/14684520510598066","article-title":"Google scholar: the pros and the cons","volume":"29","author":"Jacs\u00f3","year":"2005","journal-title":"Online Inf. Rev."},{"key":"mlstada47fbib72","article-title":"Large language model for causal decision making","author":"Jiang","year":"2024"},{"key":"mlstada47fbib73","first-page":"31038","article-title":"CLadder: assessing causal reasoning in language models","volume":"vol 36","author":"Jin","year":"2023"},{"key":"mlstada47fbib74","article-title":"Can large language models infer causation from correlation?","author":"Jin","year":"2024"},{"key":"mlstada47fbib75","author":"Jurafsky","year":"2025","edition":"3rd edn"},{"key":"mlstada47fbib76","doi-asserted-by":"publisher","DOI":"10.1201\/9781003143376)","author":"Karpatne","year":"2022","edition":"1st edn"},{"key":"mlstada47fbib77","article-title":"Evaluating interventional reasoning capabilities of large language models","author":"Kasetty","year":"2024"},{"key":"mlstada47fbib78","article-title":"ALCM: autonomous LLM-augmented causal discovery framework","author":"Khatibi","year":"2024"},{"key":"mlstada47fbib79","article-title":"Causal reasoning and large language models: opening a new frontier for causality","author":"K\u0131c\u0131man","year":"2023"},{"key":"mlstada47fbib80","doi-asserted-by":"publisher","first-page":"1912","DOI":"10.1038\/s41467-022-29543-7","article-title":"A deep learning-based hybrid model of global terrestrial evaporation","volume":"13","author":"Koppa","year":"2022","journal-title":"Nat. Commun."},{"key":"mlstada47fbib81","doi-asserted-by":"publisher","first-page":"1219","DOI":"10.1001\/jamapsychiatry.2019.1965","article-title":"Is it time to ban the P value?","volume":"76","author":"Kraemer","year":"2019","journal-title":"JAMA Psychiatry"},{"key":"mlstada47fbib82","doi-asserted-by":"publisher","first-page":"639","DOI":"10.1177\/1745691620958012","article-title":"The practical alternative to the p value is the correctly used p value","volume":"16","author":"Lakens","year":"2021","journal-title":"Perspect. Psychol. Sci."},{"key":"mlstada47fbib83","volume":"vol 17","author":"Lauritzen","year":"1996"},{"key":"mlstada47fbib84","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642428)","article-title":"Hill: a hallucination identifier for large language models","author":"Leiser","year":"2024"},{"key":"mlstada47fbib85","first-page":"pp 8871","article-title":"Context-specific causal discovery for categorical data using staged trees","volume":"vol 206","author":"Leonelli","year":"2023"},{"key":"mlstada47fbib86","first-page":"pp 9459","article-title":"Retrieval-augmented generation for knowledge-intensive NLP tasks","volume":"vol 33","author":"Lewis","year":"2020"},{"key":"mlstada47fbib87","first-page":"pp 225","article-title":"Bayesian network structure learning with side constraints","author":"Li","year":"2018"},{"key":"mlstada47fbib88","doi-asserted-by":"crossref","DOI":"10.1145\/3627673.3680042","article-title":"RealTCD: temporal causal discovery from interventional data with large language model","author":"Li","year":"2024"},{"key":"mlstada47fbib89","first-page":"pp 13201","article-title":"Prompting large language models for counterfactual generation: an empirical study","author":"Li","year":"2024"},{"key":"mlstada47fbib90","article-title":"Unveiling the pitfalls of knowledge editing for large language models","author":"Li","year":"2024"},{"key":"mlstada47fbib91","doi-asserted-by":"publisher","first-page":"9273","DOI":"10.1038\/s41598-022-12847-5","article-title":"How experts\u2019 own inconsistency relates to their confidence and between-expert disagreement","volume":"12","author":"Litvinova","year":"2022","journal-title":"Sci. Rep."},{"key":"mlstada47fbib92","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1162\/tacl_a_00638","article-title":"Lost in the middle: how language models use long contexts","volume":"12","author":"Liu","year":"2024","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"mlstada47fbib93","article-title":"Large language models to enhance Bayesian optimization","author":"Liu","year":"2024"},{"key":"mlstada47fbib94","article-title":"Can large language models build causal graphs?","author":"Long","year":"2023"},{"key":"mlstada47fbib95","article-title":"Causal discovery with language models as imperfect experts","author":"Long","year":"2023"},{"key":"mlstada47fbib96","doi-asserted-by":"publisher","first-page":"751","DOI":"10.1093\/qje\/qjad055","article-title":"Machine learning as a tool for hypothesis generation","volume":"139","author":"Ludwig","year":"2024","journal-title":"Q. J. Econ."},{"key":"mlstada47fbib97","article-title":"Self-contradictory hallucinations of large language models: evaluation, detection and mitigation","author":"M\u00fcndler","year":"2024"},{"key":"mlstada47fbib98","first-page":"pp 23","article-title":"Causal structure learning from multivariate time series in settings with unmeasured confounding","volume":"vol 92","author":"Malinsky","year":"2018"},{"key":"mlstada47fbib99","doi-asserted-by":"publisher","first-page":"1168","DOI":"10.1016\/j.ijar.2013.03.009","article-title":"An interactive approach for Bayesian network learning using domain\/expert knowledge","volume":"54","author":"Masegosa","year":"2013","journal-title":"Int. J. Approx. Reason."},{"key":"mlstada47fbib100","doi-asserted-by":"publisher","first-page":"814","DOI":"10.1108\/14684520710841784","article-title":"An exploratory study of google scholar","volume":"31","author":"Mayr","year":"2007","journal-title":"Online Inf. Rev."},{"key":"mlstada47fbib101","doi-asserted-by":"publisher","first-page":"pp 403","DOI":"10.5555\/2074158.2074204","article-title":"Causal inference and causal explanation with background knowledge","author":"Meek","year":"1995"},{"key":"mlstada47fbib102","article-title":"Graphical models: selecting causal and statistical models","author":"Meek","year":"1997"},{"key":"mlstada47fbib103","first-page":"pp 17359","article-title":"Locating and editing factual associations in GPT","volume":"vol 35","author":"Meng","year":"2022"},{"key":"mlstada47fbib104","article-title":"The impact of large language models on scientific discovery: a preliminary study using GPT-4","author":"Microsoft Research AI4Science and Microsoft Azure Quantum","year":"2023"},{"key":"mlstada47fbib105","article-title":"Beyond accuracy: evaluating self-consistency of code large language models with IdentityChain","author":"Min","year":"2024"},{"key":"mlstada47fbib106","first-page":"pp 10","article-title":"Annotating causality in the TempEval-3 corpus","author":"Mirza","year":"2014"},{"key":"mlstada47fbib107","article-title":"Fast model editing at scale","author":"Mitchell","year":"2022"},{"key":"mlstada47fbib108","article-title":"Tree of uncertain thoughts reasoning for large language models","author":"Mo","year":"2023"},{"key":"mlstada47fbib109","doi-asserted-by":"publisher","first-page":"539","DOI":"10.1214\/19-AOS1821","article-title":"Markov equivalence of marginalized local independence graphs","volume":"48","author":"Mogensen","year":"2020","journal-title":"Ann. Stat."},{"key":"mlstada47fbib110","first-page":"1","article-title":"Distinguishing cause from effect using observational data: methods and benchmarks","volume":"17","author":"Mooij","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"mlstada47fbib111","doi-asserted-by":"publisher","first-page":"14313","DOI":"10.1073\/pnas.0802272105","article-title":"Network inference using informative priors","volume":"105","author":"Mukherjee","year":"2008","journal-title":"Proc. Natl Acad. Sci."},{"key":"mlstada47fbib112","article-title":"Answering causal questions with augmented LLMs","author":"Pawlowski","year":"2023"},{"key":"mlstada47fbib113","first-page":"p 3","volume":"vol 19","author":"Pearl","year":"2000"},{"key":"mlstada47fbib114","doi-asserted-by":"publisher","first-page":"771","DOI":"10.1162\/NECO_a_00708","article-title":"Structural intervention distance for evaluating causal graphs","volume":"27","author":"Peters","year":"2015","journal-title":"Neural Comput."},{"key":"mlstada47fbib115","doi-asserted-by":"publisher","first-page":"pp 671","DOI":"10.1145\/3501714.3501752","article-title":"Causal models for dynamical systems","author":"Peters","year":"2022"},{"key":"mlstada47fbib116","author":"Peters","year":"2017"},{"key":"mlstada47fbib117","first-page":"pp 589","article-title":"Identifiability of causal graphs using functional models","author":"Peters","year":"2011"},{"key":"mlstada47fbib118","article-title":"Narrating causal graphs with large language models","author":"Phatak","year":"2024"},{"key":"mlstada47fbib119","author":"Popper","year":"2005"},{"key":"mlstada47fbib120","first-page":"pp 1593","article-title":"Robust knowledge extraction from large language models using social choice theory","author":"Potyka","year":"2024"},{"key":"mlstada47fbib121","doi-asserted-by":"publisher","first-page":"686","DOI":"10.1016\/j.jcp.2018.10.045","article-title":"Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations","volume":"378","author":"Raissi","year":"2019","journal-title":"J. Comput. Phys."},{"key":"mlstada47fbib122","article-title":"A comparison of public causal search packages on linear, Gaussian data with no latent variables","author":"Ramsey","year":"2017"},{"key":"mlstada47fbib123","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1038\/s41586-019-0912-1","article-title":"Deep learning and process understanding for data-driven earth system science","volume":"566","author":"Reichstein","year":"2019","journal-title":"Nature"},{"key":"mlstada47fbib124","first-page":"pp 27772","article-title":"Beware of the simulated dag! Causal discovery benchmarks may be easy to game","volume":"vol 34","author":"Reisach","year":"2021"},{"key":"mlstada47fbib125","article-title":"Choice of plausible alternatives: an evaluation of commonsense causal reasoning","author":"Roemmele","year":"2011"},{"key":"mlstada47fbib126","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2023.emnlp-main.940","article-title":"CRAB: assessing the strength of causal relationships between real-world events","author":"Romanou","year":"2023"},{"key":"mlstada47fbib127","doi-asserted-by":"publisher","first-page":"2553","DOI":"10.1038\/s41467-019-10105-3","article-title":"Inferring causation from time series in earth system sciences","volume":"10","author":"Runge","year":"2019","journal-title":"Nat. Commun."},{"key":"mlstada47fbib128","doi-asserted-by":"publisher","first-page":"1917","DOI":"10.1038\/s41467-023-37546-1","article-title":"Modern causal inference approaches to investigate biodiversity-ecosystem functioning relationships","volume":"14","author":"Runge","year":"2023","journal-title":"Nat. Commun."},{"key":"mlstada47fbib129","doi-asserted-by":"publisher","first-page":"487","DOI":"10.1038\/s43017-023-00431-y","article-title":"Causal inference for time series","volume":"4","author":"Runge","year":"2023","journal-title":"Nat. Rev. Earth Environ."},{"key":"mlstada47fbib130","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"mlstada47fbib131","doi-asserted-by":"publisher","first-page":"953","DOI":"10.1080\/10705511.2022.2056039","article-title":"The challenge of generating causal hypotheses using network models","volume":"29","author":"Ryan","year":"2022","journal-title":"Struct. Equ. Model."},{"key":"mlstada47fbib132","doi-asserted-by":"publisher","first-page":"523","DOI":"10.1126\/science.1105809","article-title":"Causal protein-signaling networks derived from multiparameter single-cell data","volume":"308","author":"Sachs","year":"2005","journal-title":"Science"},{"key":"mlstada47fbib133","author":"Scutari","year":"2014"},{"key":"mlstada47fbib134","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1016\/j.ijar.2019.10.003","article-title":"Who learns better Bayesian network structures: accuracy and speed of structure learning algorithms","volume":"115","author":"Scutari","year":"2019","journal-title":"Int. J. Approx. Reason."},{"key":"mlstada47fbib135","doi-asserted-by":"publisher","first-page":"1514","DOI":"10.1214\/19-AOS1857","article-title":"The hardness of conditional independence testing and the generalised covariance measure","volume":"48","author":"Shah","year":"2018","journal-title":"Ann. Stat."},{"key":"mlstada47fbib136","author":"Shavit"},{"key":"mlstada47fbib137","author":"Spirtes","year":"2001"},{"key":"mlstada47fbib138","article-title":"Beyond the imitation game: quantifying and extrapolating the capabilities of language models","author":"Srivastava","year":"2023","journal-title":"Trans. Mach. Learn. Res."},{"key":"mlstada47fbib139","doi-asserted-by":"publisher","DOI":"10.1038\/srep21691","article-title":"On the causal structure between CO2 and global temperature","volume":"6","author":"Stips","year":"2016","journal-title":"Sci. Rep."},{"key":"mlstada47fbib140","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2024.naacl-long.18","article-title":"Head-to-tail: how knowledgeable are large language models (LLMs)? A.K.A. will LLMs replace knowledge graphs?","author":"Sun","year":"2024"},{"key":"mlstada47fbib141","first-page":"pp 2178","article-title":"Benchmarking hallucination in large language models based on unanswerable math word problem","author":"Sun","year":"2024"},{"key":"mlstada47fbib142","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v105.i09","article-title":"Bayesian structure learning and sampling of Bayesian networks with the R package BiDAG","volume":"105","author":"Suter","year":"2023","journal-title":"J. Stat. Softw."},{"key":"mlstada47fbib143","first-page":"pp 2327","article-title":"Found in the middle: permutation self-consistency improves listwise ranking in large language models","author":"Tang","year":"2024"},{"key":"mlstada47fbib144","first-page":"pp 1072","article-title":"Quantifying uncertainty in natural language explanations of large language models","author":"Tanneru","year":"2024"},{"key":"mlstada47fbib145","doi-asserted-by":"publisher","first-page":"889","DOI":"10.1016\/j.artint.2010.05.004","article-title":"Causal analysis with chain event graphs","volume":"174","author":"Thwaites","year":"2010","journal-title":"Artif. Intell."},{"key":"mlstada47fbib146","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2023.emnlp-main.330","article-title":"Just ask for calibration: strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback","author":"Tian","year":"2023"},{"key":"mlstada47fbib147","doi-asserted-by":"publisher","DOI":"10.31234\/osf.io\/7ck9m)","article-title":"Automating psychological hypothesis generation with AI: large language models meet causal graph","author":"Tong","year":"2023"},{"key":"mlstada47fbib148","doi-asserted-by":"publisher","first-page":"5235","DOI":"10.1111\/gcb.15203","article-title":"Partitioning net carbon dioxide fluxes into photosynthesis and respiration using neural networks","volume":"26","author":"Tramontana","year":"2020","journal-title":"Glob. Change Biol."},{"key":"mlstada47fbib149","first-page":"pp 12793","article-title":"Neuropathic pain diagnosis simulator for causal discovery algorithm evaluation","volume":"vol 32","author":"Tu","year":"2019"},{"key":"mlstada47fbib150","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1186\/1471-2105-7-43","article-title":"SynTReN: a generator of synthetic gene expression data for design and analysis of structure learning algorithms","volume":"7","author":"Van den Bulcke","year":"2006","journal-title":"BMC Bioinform."},{"key":"mlstada47fbib151","first-page":"pp 989","article-title":"Graphical continuous Lyapunov models","volume":"vol 124","author":"Varando","year":"2020"},{"key":"mlstada47fbib152","article-title":"Causal inference using LLM-guided discovery","author":"Vashishtha","year":"2023"},{"key":"mlstada47fbib153","article-title":"DoubleLingo: causal estimation with large language models","author":"Veljanovski","year":"2023"},{"key":"mlstada47fbib154","doi-asserted-by":"publisher","first-page":"35","DOI":"10.3758\/s13423-017-1343-3","article-title":"Bayesian inference for psychology. Part I: theoretical advantages and practical ramifications","volume":"25","author":"Wagenmakers","year":"2018","journal-title":"Psychon. Bull. Rev."},{"key":"mlstada47fbib155","article-title":"Bridging causal discovery and large language models: a comprehensive survey of integrative approaches and future directions","author":"Wan","year":"2024"},{"key":"mlstada47fbib156","article-title":"Hypothesis search: inductive reasoning with language models","author":"Wang","year":"2024"},{"key":"mlstada47fbib157","first-page":"pp 12288","article-title":"No need for large-scale search: exploring large language models in complex knowledge base question answering","author":"Wang","year":"2024"},{"key":"mlstada47fbib158","article-title":"Assessing the reliability of large language model knowledge","author":"Wang","year":"2023"},{"key":"mlstada47fbib159","first-page":"pp 926","article-title":"MAVEN-ERE: a unified large-scale dataset for event coreference, temporal, causal and subevent relation extraction","author":"Wang","year":"2022"},{"key":"mlstada47fbib160","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2020.106515","article-title":"Learning Bayesian networks based on order graph with ancestral constraints","volume":"211","author":"Wang","year":"2021","journal-title":"Knowl.-Based Syst."},{"key":"mlstada47fbib161","author":"Wasserman","year":"2013"},{"key":"mlstada47fbib162","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/00031305.2019.1583913","article-title":"Moving to a world beyond \u201cp < 0.05\u201d","volume":"73","author":"Wasserstein","year":"2019","journal-title":"Am. Stat."},{"key":"mlstada47fbib163","article-title":"Chain-of-thought prompting elicits reasoning in large language models","author":"Wei","year":"2023"},{"key":"mlstada47fbib164","doi-asserted-by":"publisher","first-page":"15","DOI":"10.2202\/1544-6115.1282","article-title":"Reconstructing gene regulatory networks with Bayesian networks by combining expression data with multiple sources of prior knowledge","volume":"6","author":"Werhli","year":"2007","journal-title":"Stat. Appl. Genet. Mol. Biol."},{"key":"mlstada47fbib165","doi-asserted-by":"publisher","author":"Wolfschwenger","year":"2021","DOI":"10.1093\/hepl\/9780198850298.003.0042)"},{"key":"mlstada47fbib166","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1214\/aoms\/1177732676","article-title":"The method of path coefficients","volume":"5","author":"Wright","year":"1934","journal-title":"Ann. Math. Stat."},{"key":"mlstada47fbib167","article-title":"Can LLMs express their uncertainty? An empirical evaluation of confidence elicitation in LLMs","author":"Xiong","year":"2024"},{"key":"mlstada47fbib168","first-page":"pp 11956","article-title":"Multi-perspective improvement of knowledge graph completion with large language models","author":"Xu","year":"2024"},{"key":"mlstada47fbib169","article-title":"Are large language models really good logical reasoners? A comprehensive evaluation and beyond","author":"Xu","year":"2023"},{"key":"mlstada47fbib170","doi-asserted-by":"publisher","first-page":"713","DOI":"10.1007\/s11390-015-1556-8","article-title":"A structure learning algorithm for Bayesian network using prior knowledge","volume":"30","author":"Xu","year":"2015","journal-title":"J. Comput. Sci. Technol."},{"key":"mlstada47fbib171","article-title":"Large language models as optimizers","author":"Yang","year":"2024"},{"key":"mlstada47fbib172","article-title":"A critical review of causal inference benchmarks for large language models","author":"Yang","year":"2023"},{"key":"mlstada47fbib173","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2020.109913","article-title":"B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data","volume":"425","author":"Yang","year":"2021","journal-title":"J. Comput. Phys."},{"key":"mlstada47fbib174","first-page":"11809","article-title":"Tree of thoughts: deliberate problem solving with large language models","volume":"vol 36","author":"Yao","year":"2024"},{"key":"mlstada47fbib175","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2023.emnlp-main.632","article-title":"Editing large language models: problems, methods and opportunities","author":"Yao","year":"2023"},{"key":"mlstada47fbib176","first-page":"pp 609","article-title":"Aggregation of reasoning: a hierarchical framework for enhancing answer selection in large language models","author":"Yin","year":"2024"},{"key":"mlstada47fbib177","article-title":"KoLA: carefully benchmarking world knowledge of large language models","author":"Yu","year":"2024"},{"key":"mlstada47fbib178","article-title":"Causal parrots: Large language models may talk causality but are not causal","author":"Ze\u010devi\u0107","year":"2023","journal-title":"Trans. Mach. Learn. Res."},{"key":"mlstada47fbib179","article-title":"Understanding causality with large language models: feasibility and opportunities","author":"Zhang","year":"2023"},{"key":"mlstada47fbib180","first-page":"pp 1","article-title":"Causality in the time of LLMs: round table discussion results of clear 2023","author":"Zhang","year":"2023"},{"key":"mlstada47fbib181","first-page":"pp 19560","article-title":"Tree-of-reasoning question decomposition for complex question answering with large language models","volume":"vol 38","author":"Zhang","year":"2024"},{"key":"mlstada47fbib182","article-title":"Causal graph discovery with retrieval-augmented generation based large language models","author":"Zhang","year":"2024"},{"key":"mlstada47fbib183","doi-asserted-by":"publisher","first-page":"14496","DOI":"10.1029\/2019GL085291","article-title":"Physics-constrained machine learning of evapotranspiration","volume":"46","author":"Zhao","year":"2019","journal-title":"Geophys. Res. Lett."},{"key":"mlstada47fbib184","first-page":"pp 8931","article-title":"Improving the robustness of large language models via consistency alignment","author":"Zhao","year":"2024"},{"key":"mlstada47fbib185","first-page":"pp 14103","article-title":"Revisiting the self-consistency challenges in multi-choice question formats for large language model evaluation","author":"Zhou","year":"2024"},{"key":"mlstada47fbib186","article-title":"CausalBench: a comprehensive benchmark for causal learning capability of large language models","author":"Zhou","year":"2024"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ada47f","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ada47f\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ada47f","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ada47f\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ada47f\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ada47f\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ada47f\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ada47f\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,31]],"date-time":"2025-01-31T08:29:11Z","timestamp":1738312151000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ada47f"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,31]]},"references-count":186,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,1,31]]},"published-print":{"date-parts":[[2025,3,31]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/ada47f","relation":{"has-review":[{"id-type":"doi","id":"10.1088\/2632-2153\/ADA47F\/v1\/review3","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2632-2153\/ADA47F\/v1\/review2","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2632-2153\/ADA47F\/v1\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2632-2153\/ADA47F\/v2\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2632-2153\/ADA47F\/v2\/response1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2632-2153\/ADA47F\/v1\/decision1","asserted-by":"object"}]},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,31]]},"assertion":[{"value":"Large language models for causal hypothesis generation in science","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2025 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2024-06-16","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-12-30","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-01-31","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}