{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T14:45:15Z","timestamp":1787582715725,"version":"build-2736575974"},"reference-count":75,"publisher":"The Royal Society","issue":"2324","license":[{"start":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T00:00:00Z","timestamp":1784160000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,7,16]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>This paper explores the prospect of a scientific computing chatbot (SciCompBot): an artificial intelligence (AI) system designed to address advanced problems in optimization, spectral computations (eigenvalues, eigenvectors, spectra), differential equations, inverse problems and related areas. With powerful chatbots like ChatGPT-5.2, it is no longer science fiction for AI to accept these problems; ChatGPT-5.2 does so willingly. Yet, current chatbots suffer from a critical flaw: they often produce incorrect answers with unwarranted confidence, making them fundamentally&amp;gt; untrustworthy for scientific computing. We investigate the mathematical challenges involved in developing a trustworthy SciCompBot and demonstrate that tools from the foundations of computational mathematics, particularly the solvability complexity index (SCI) hierarchy, are necessary in this endeavour. Our results present new strategies for overcoming central obstacles for building a trustworthy SciCompBot: the issues of non-computability and the phenomenon of generalized hardness of approximation (GHA). We identify a key feature that a trustworthy SciCompBot must have: it must be \u2018chatty\u2019. That is, it must initiate a dialogue with the user. This work fits in the broader programme of ensuring trustworthiness and safety in AI systems, which is now a major topic of interest owing to explosive growth in AI development in recent years.<\/jats:p>\n                  <jats:p>This article is part of the theme issue \u2018Safe, secure and robust AI for safety-critical systems\u2019.<\/jats:p>","DOI":"10.1098\/rsta.2025.0116","type":"journal-article","created":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T23:39:29Z","timestamp":1784245169000},"source":"Crossref","is-referenced-by-count":0,"title":["Trustworthiness in AI: on SciCompBot\u2014the scientific computing chatbot"],"prefix":"10.1098","volume":"384","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2700-9446","authenticated-orcid":false,"given":"Anders Christian","family":"Hansen","sequence":"first","affiliation":[{"name":"Department of Applied Mathematics and Theoretical Physics, University of Cambridge , Cambridge, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fabian","family":"Circelli","sequence":"additional","affiliation":[{"name":"Department of Applied Mathematics and Theoretical Physics, University of Cambridge , Cambridge, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"175","published-online":{"date-parts":[[2026,7,16]]},"reference":[{"key":"2026082410124910100_bib1","doi-asserted-by":"crossref","first-page":"860","DOI":"10.1137\/18M1165748","article-title":"Deep learning: an introduction for applied mathematicians","volume":"61","author":"Higham","year":"2019","journal-title":"SIAM Rev."},{"key":"2026082410124910100_bib2","article-title":"Sparks of artificial general intelligence: early experiments with GPT-4","author":"Bubeck","year":"2023","journal-title":"arXiv"},{"key":"2026082410124910100_bib3","article-title":"GPT-4 technical report","author":"OpenAI","year":"2024","journal-title":"arXiv"},{"key":"2026082410124910100_bib4","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1038\/d41586-023-00056-7","article-title":"Abstracts written by ChatGPT fool scientists","volume":"613","author":"Else","year":"2023","journal-title":"Nature"},{"key":"2026082410124910100_bib5","article-title":"Gemini: a family of highly capable multimodal models","author":"Gemini Team Google","year":"2024","journal-title":"arXiv"},{"key":"2026082410124910100_bib6","doi-asserted-by":"crossref","DOI":"10.1017\/9781009089517","volume-title":"Data-driven science and engineering: machine learning, dynamical systems, and control","author":"Brunton","year":"2022","edition":"2nd"},{"key":"2026082410124910100_bib7","doi-asserted-by":"crossref","first-page":"22445","DOI":"10.1073\/pnas.1906995116","article-title":"Data-driven discovery of coordinates and governing equations","volume":"116","author":"Champion","year":"2019","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2026082410124910100_bib8","first-page":"230","article-title":"On computable numbers, with an application to the entscheidungsproblem","volume":"42","author":"Turing","year":"1936","journal-title":"Proc. Lond. Math. Soc. (2)"},{"key":"2026082410124910100_bib9","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1007\/BF01700692","article-title":"\u00dcber formal unentscheidbare S\u00e4tze der principia mathematica und verwandter systeme","volume":"38","author":"G\u00f6del","year":"1931","journal-title":"Monatshefte f\u00fcr mathematik und physik"},{"key":"2026082410124910100_bib10","volume-title":"G\u00f6del\u2019s theorem: an incomplete guide to its use and abuse","author":"Franz\u00e9n","year":"2005"},{"key":"2026082410124910100_bib11","volume-title":"Hallucinations could blunt ChatGPT\u2019s success","author":"Smith","year":"2023"},{"key":"2026082410124910100_bib12","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1145\/3571730","article-title":"Survey of hallucination in natural language generation","volume":"55","author":"Ji","year":"2023","journal-title":"ACM Comput. Surv."},{"key":"2026082410124910100_bib13","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1038\/s41586-024-07421-0","article-title":"Detecting hallucinations in large language models using semantic entropy","volume":"630","author":"Farquhar","year":"2024","journal-title":"Nature"},{"key":"2026082410124910100_bib14","article-title":"On hallucination and predictive uncertainty in conditional language generation","author":"Xiao","year":"2021","journal-title":"arXiv"},{"key":"2026082410124910100_bib15","first-page":"675","article-title":"A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity","volume-title":"Proc. 13th Int. Joint Conf. on Natural Language Processing and the 3rd Conf. of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Bang","year":"2023"},{"key":"2026082410124910100_bib16","article-title":"\u2018Hallucinate\u2019 chosen as Cambridge dictionary\u2019s word of the year","author":"Creamer","year":"2023","journal-title":"The Guardian"},{"key":"2026082410124910100_bib17","article-title":"Navigating the jagged technological frontier: field experimental evidence of the effects of AI on knowledge worker productivity and quality","author":"Dell\u2019Acqua","year":"2023","journal-title":"Working paper"},{"key":"2026082410124910100_bib18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/IJCNN48605.2020.9207472","article-title":"On adversarial examples and stealth attacks in artificial intelligence systems","volume-title":"2020 Int. Joint Conf. on Neural Networks (IJCNN)","author":"Tyukin","year":"2020"},{"key":"2026082410124910100_bib19","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1093\/imamat\/hxad017","article-title":"Adversarial ink: componentwise backward error attacks on deep learning","volume":"89","author":"Beerens","year":"2023","journal-title":"IMA J. Appl. Math."},{"key":"2026082410124910100_bib20","article-title":"The mathematics of reliable artificial intelligence","volume":"57","author":"Kutyniok","year":"2024","journal-title":"SIAM News"},{"key":"2026082410124910100_bib21","doi-asserted-by":"crossref","first-page":"20752","DOI":"10.1038\/s41598-024-71761-0","article-title":"Trust, trustworthiness and AI governance","volume":"14","author":"Lahusen","year":"2024","journal-title":"Sci. Rep"},{"key":"2026082410124910100_bib22","first-page":"39","article-title":"Trustworthy artificial intelligence: a review","volume":"55","author":"Kaur","year":"2022","journal-title":"ACM Comput. Surv"},{"key":"2026082410124910100_bib23","article-title":"When can you trust feature selection?\u2014I: a condition-based analysis of LASSO and generalised hardness of approximation","author":"Bastounis","year":"2023","journal-title":"arXiv"},{"key":"2026082410124910100_bib24","article-title":"The extended Smale\u2019s 9th problem\u2014on computational barriers and paradoxes in estimation, regularisation, computer-assisted proofs and learning","author":"Bastounis","year":"2022","journal-title":"arXiv"},{"key":"2026082410124910100_bib25","article-title":"Forthcoming Generalised hardness of approximation and the SCI hierarchy\u2014on determining the boundaries of training algorithms in AI","author":"Gazdag","journal-title":"Found. Computat. Math."},{"key":"2026082410124910100_bib26","doi-asserted-by":"crossref","first-page":"804","DOI":"10.1137\/S0097539796302531","article-title":"Free bits, PCPs, and nonapproximability\u2014towards tight results","volume":"27","author":"Bellare","year":"1998","journal-title":"SIAM J. Comput."},{"key":"2026082410124910100_bib27","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1145\/278298.278306","article-title":"Proof verification and the hardness of approximation problems","volume":"45","author":"Arora","year":"1998","journal-title":"J. ACM"},{"key":"2026082410124910100_bib28","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1007\/BF02392825","article-title":"Clique is hard to approximate within n1-\u03f5","volume":"182","author":"H\u00e5stad","year":"1999","journal-title":"Acta Math."},{"key":"2026082410124910100_bib29","doi-asserted-by":"crossref","first-page":"e2107151119","DOI":"10.1073\/pnas.2107151119","article-title":"The difficulty of computing stable and accurate neural networks: on the barriers of deep learning and Smale\u2019s 18th problem","volume":"119","author":"Colbrook","year":"2022","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2026082410124910100_bib30","doi-asserted-by":"crossref","first-page":"697","DOI":"10.1007\/s10208-021-09509-9","article-title":"Computing the sound of the sea in a seashell","volume":"22","author":"Ben-Artzi","year":"2022","journal-title":"Found. Comput. Math."},{"key":"2026082410124910100_bib31","article-title":"Computing spectra\u2014on the solvability complexity index hierarchy and towers of algorithms","author":"Ben-Artzi","year":"2020","journal-title":"arXiv"},{"key":"2026082410124910100_bib32","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1090\/S0894-0347-2010-00676-5","article-title":"On the solvability complexity index, the n-pseudospectrum and approximations of spectra of operators","volume":"24","author":"Hansen","year":"2011","journal-title":"J. Am. Math. Soc."},{"key":"2026082410124910100_bib33","doi-asserted-by":"crossref","first-page":"723","DOI":"10.1007\/s10208-022-09598-0","article-title":"On the computation of geometric features of spectra of linear operators on Hilbert spaces","volume":"24","author":"Colbrook","year":"2024","journal-title":"Found. Comp. Math."},{"key":"2026082410124910100_bib34","doi-asserted-by":"crossref","first-page":"931","DOI":"10.1016\/j.crma.2015.08.002","article-title":"New barriers in complexity theory: on the solvability complexity index and the towers of algorithms","volume":"353","author":"Ben-Artzi","year":"2015","journal-title":"C. R. Math."},{"key":"2026082410124910100_bib35","doi-asserted-by":"crossref","first-page":"3633","DOI":"10.4171\/jems\/1258","article-title":"Computing scattering resonances","volume":"25","author":"Ben-Artzi","year":"2022","journal-title":"J. Eur. Math. Soc."},{"key":"2026082410124910100_bib36","doi-asserted-by":"crossref","DOI":"10.1201\/b18401","volume-title":"Statistical learning with sparsity: the Lasso and generalizations","author":"Hastie","year":"2015"},{"key":"2026082410124910100_bib37","volume-title":"Turing computability: theory and applications","author":"Soare","year":"2016","edition":"1st"},{"key":"2026082410124910100_bib38","doi-asserted-by":"crossref","DOI":"10.1017\/9781108377447","volume-title":"Compressive imaging: structure, sampling, learning","author":"Adcock","year":"2021"},{"key":"2026082410124910100_bib39","article-title":"Alice in wonderland: simple tasks showing complete reasoning breakdown in state-of-the-art large language models","author":"Nezhurina","year":"2025","journal-title":"arXiv"},{"key":"2026082410124910100_bib40","article-title":"Easy problems that LLMs get wrong","author":"Williams","year":"2024","journal-title":"arXiv"},{"key":"2026082410124910100_bib41","doi-asserted-by":"crossref","first-page":"28083","DOI":"10.1038\/s41598-024-79531-8","article-title":"Testing AI on language comprehension tasks reveals insensitivity to underlying meaning","volume":"14","author":"Dentella","year":"2024","journal-title":"Sci. Rep"},{"key":"2026082410124910100_bib42","article-title":"On the consistent reasoning paradox of intelligence and optimal trust in AI: the power of \u2018I don\u2019t know\u2019","author":"Bastounis","year":"2024","journal-title":"arXiv"},{"key":"2026082410124910100_bib43","volume-title":"Computational mathematics in computer assisted proofs","author":"Fefferman","year":"2022"},{"key":"2026082410124910100_bib44","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511804441","volume-title":"Convex optimization","author":"Boyd","year":"2004"},{"key":"2026082410124910100_bib45","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-38896-5","volume-title":"Condition: the geometry of numerical algorithms. Grundlehren der mathematischen Wissenschaften","author":"B\u00fcrgisser","year":"2013"},{"key":"2026082410124910100_bib46","volume-title":"Interior-point polynomial algorithms in convex programming. SIAM studies in applied mathematics","author":"Nesterov","year":"1994"},{"key":"2026082410124910100_bib47","doi-asserted-by":"crossref","DOI":"10.1137\/1.9780898718829","volume-title":"Lectures on modern convex optimization: analysis, algorithms, and engineering applications","author":"Ben-Tal","year":"2001"},{"key":"2026082410124910100_bib48","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1090\/S0273-0979-04-01040-7","article-title":"The interior-point revolution in optimization: history, recent developments, and lasting consequences","volume":"42","author":"Wright","year":"2005","journal-title":"Bull. Am. Math. Soc."},{"key":"2026082410124910100_bib49","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1137\/110830150","article-title":"Convergence of the restricted Nelder\u2013Mead algorithm in two dimensions","volume":"22","author":"Lagarias","year":"2012","journal-title":"SIAM J. Optim."},{"key":"2026082410124910100_bib50","volume-title":"Numerical optimization","author":"Nocedal","year":"2006","edition":"2nd"},{"key":"2026082410124910100_bib51","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4171\/jems\/1289","article-title":"The foundations of spectral computations via the Solvability Complexity Index hierarchy","volume":"25","author":"Colbrook","year":"2022","journal-title":"J. Eur. Math. Soc."},{"key":"2026082410124910100_bib52","volume-title":"Forthcoming (August 2026) Infinite-dimensional spectral computations","author":"Colbrook"},{"key":"2026082410124910100_bib53","volume-title":"Complexity and real computation","author":"Blum","year":"1997"},{"key":"2026082410124910100_bib54","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1090\/S0273-0979-1989-15750-9","article-title":"On a theory of computation and complexity over the real numbers: NP-completeness, recursive functions and universal machines","volume":"21","author":"Blum","year":"1989","journal-title":"Bull. Am. Math. Soc."},{"key":"2026082410124910100_bib55","volume-title":"Classical recursion theory: the theory of functions and sets of natural numbers","author":"Odifreddi","year":"1992"},{"key":"2026082410124910100_bib56","first-page":"464","article-title":"On the computability of AIXI","volume-title":"Proc. 31st Conf. on Uncertainty in Artificial Intelligence UAI\u201915","author":"Leike","year":"2015"},{"key":"2026082410124910100_bib57","doi-asserted-by":"crossref","DOI":"10.1007\/b138233","volume-title":"Universal artificial intelligence: sequential decisions based on algorithmic probability","author":"Hutter","year":"2005"},{"key":"2026082410124910100_bib58","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511804090","volume-title":"Computational complexity: a modern approach","author":"Arora","year":"2009"},{"key":"2026082410124910100_bib59","article-title":"When can you trust feature selection?\u2014II: on the effects of random data on condition in statistics and optimisation","author":"Bastounis","year":"2023","journal-title":"arXiv"},{"key":"2026082410124910100_bib60","first-page":"114","article-title":"The role of C*-algebras in infinite-dimensional numerical linear algebra","volume-title":"C*-algebras: 1943\u20131993 (San Antonio, TX, 1993), Contemp. Math.","author":"Arveson","year":"1994"},{"key":"2026082410124910100_bib61","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1006\/jfan.1994.1072","article-title":"C*-algebras and numerical linear algebra","volume":"122","author":"Arveson","year":"1994","journal-title":"J. Funct. Anal."},{"key":"2026082410124910100_bib62","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1145\/320954.320960","article-title":"The Jacobi method for real symmetric matrices","volume":"6","author":"Goldstine","year":"1959","journal-title":"J. ACM"},{"key":"2026082410124910100_bib63","doi-asserted-by":"crossref","first-page":"474","DOI":"10.1073\/pnas.34.10.474","article-title":"Rayleigh-Ritz and A. Weinstein methods for approximation of eigenvalues: I. operations in a Hilbert space","volume":"34","author":"Aronszajn","year":"1948","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2026082410124910100_bib64","first-page":"1","article-title":"Infinite matrices and projection methods","volume-title":"Lectures on operator theory and its applications (Waterloo, ON, 1994), Fields Inst. Monogr.","author":"B\u00f6ttcher","year":"1996"},{"key":"2026082410124910100_bib65","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4612-1426-7","volume-title":"Introduction to large truncated Toeplitz matrices","author":"B\u00f6ttcher","year":"1999"},{"key":"2026082410124910100_bib66","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1090\/S0025-5718-06-01893-X","article-title":"Quasi-diagonality and the finite section method","volume":"76","author":"Brown","year":"2007","journal-title":"Math. Comput."},{"key":"2026082410124910100_bib67","doi-asserted-by":"crossref","first-page":"S67","DOI":"10.1002\/cpa.3160390707","article-title":"The n-body problem in quantum mechanics","volume":"39","author":"Fefferman","year":"1986","journal-title":"Commun. Pure Appl. Math."},{"key":"2026082410124910100_bib68","doi-asserted-by":"crossref","first-page":"6055","DOI":"10.1073\/pnas.76.12.6055","article-title":"On the lowest eigenvalue of a pseudo-differential operator","volume":"76","author":"Fefferman","year":"1979","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2026082410124910100_bib69","doi-asserted-by":"crossref","first-page":"621","DOI":"10.1142\/S0129055X94000213","article-title":"Finite approximations to quantum systems","volume":"6","author":"Digernes","year":"1994","journal-title":"Rev. Math. Phys."},{"key":"2026082410124910100_bib70","doi-asserted-by":"crossref","first-page":"2092","DOI":"10.1016\/j.jfa.2008.01.006","article-title":"On the approximation of spectra of linear operators on Hilbert spaces","volume":"254","author":"Hansen","year":"2008","journal-title":"J. Funct. Anal."},{"key":"2026082410124910100_bib71","volume-title":"Complexity theory of real functions","author":"Ko","year":"2012"},{"key":"2026082410124910100_bib72","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9781107325630","volume-title":"Computation and automata","author":"Salomaa","year":"1985"},{"key":"2026082410124910100_bib73","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/j.apal.2004.08.001","article-title":"A Banach-Mazur computable but not Markov computable function on the computable real numbers","volume":"132","author":"Hertling","year":"2005","journal-title":"Ann. Pure Appl. Logic"},{"key":"2026082410124910100_bib74","doi-asserted-by":"crossref","DOI":"10.52202\/068431-1800","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume-title":"Proc. 36th Int. Conf. on Neural Information Processing Systems NIPS \u201922","author":"Wei","year":"2022"},{"key":"2026082410124910100_bib75","article-title":"Towards understanding chain-of-thought prompting: an empirical study of what matters","author":"Wang","year":"2023","journal-title":"arXiv"}],"container-title":["Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/royalsocietypublishing.org\/rsta\/article-pdf\/doi\/10.1098\/rsta.2025.0116\/6161031\/rsta.2025.0116.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/royalsocietypublishing.org\/rsta\/article-pdf\/doi\/10.1098\/rsta.2025.0116\/6161031\/rsta.2025.0116.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T14:13:01Z","timestamp":1787580781000},"score":1,"resource":{"primary":{"URL":"https:\/\/royalsocietypublishing.org\/rsta\/article\/384\/2324\/20250116\/482538\/Trustworthiness-in-AI-on-SciCompBot-the-scientific"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,16]]},"references-count":75,"journal-issue":{"issue":"2324","published-online":{"date-parts":[[2026,7,16]]},"published-print":{"date-parts":[[2026,7,16]]}},"URL":"https:\/\/doi.org\/10.1098\/rsta.2025.0116","relation":{},"ISSN":["1364-503X","1471-2962"],"issn-type":[{"value":"1364-503X","type":"print"},{"value":"1471-2962","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,16]]},"article-number":"20250116"}}