{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T19:44:13Z","timestamp":1792266253402,"version":"4.4.0"},"reference-count":48,"publisher":"Oxford University Press (OUP)","issue":"10","license":[{"start":{"date-parts":[[2026,9,21]],"date-time":"2026-09-21T00:00:00Z","timestamp":1789948800000},"content-version":"vor","delay-in-days":2,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"JSPS KAKENHI","award":["23H03412"],"award-info":[{"award-number":["23H03412"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,10,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>While Protein Language Models (PLMs) fine-tuned on biophysical data achieve high predictive accuracy, the physical principles underlying their predictions remain obscure. Deciphering these representations offers a unique opportunity to not only interpret model decisions but also to discover novel biophysical insights governing protein properties. Here, we present a framework using Sparse Autoencoders (SAEs) to extract mechanistic knowledge from PLMs fine-tuned for nanobody thermostability.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We fine-tuned the ESM-2 model on the nanobody thermostability dataset, achieving superior performance compared to significantly larger state-of-the-art models. SAE analysis successfully decomposed the model\u2019s dense embeddings into sparse, interpretable features without loss of predictive accuracy. We characterized these features through both global and local analyses. Global analysis provided an aggregate map of position-dependent feature contributions, whereas local analysis identified specific residue-level patterns, including known determinants such as the VHH-tetrad and critical disulfide bonds, as well as candidate stabilizing residues. Free Energy Perturbation calculations supported the structural plausibility of selected residue-level hypotheses. These results show that SAE-based interpretation can generate testable, structurally grounded hypotheses for rational protein engineering.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability<\/jats:title>\n                    <jats:p>The data and source code of the proposed method are available at GitHub (https:\/\/github.com\/matsunagalab\/paper_nanobody-thermostability-sae) and Zenodo (DOI: 10.5281\/zenodo.18012027).<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btag685","type":"journal-article","created":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T18:27:44Z","timestamp":1789756064000},"source":"Crossref","is-referenced-by-count":0,"title":["Mechanistic interpretability of fine-tuned protein language models for nanobody thermostability prediction"],"prefix":"10.1093","volume":"42","author":[{"given":"Taihei","family":"Murakami","sequence":"first","affiliation":[{"name":"Graduate School of Science and Engineering, Saitama University , Saitama 338-8570,","place":["Japan"]},{"name":"Research & Development Department, Epsilon Molecular Engineering, Inc , Saitama 338-8570,","place":["Japan"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuki","family":"Hashidate","sequence":"additional","affiliation":[{"name":"Graduate School of Science and Engineering, Saitama University , Saitama 338-8570,","place":["Japan"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2872-3908","authenticated-orcid":false,"given":"Yasuhiro","family":"Matsunaga","sequence":"additional","affiliation":[{"name":"Graduate School of Science and Engineering, Saitama University , Saitama 338-8570,","place":["Japan"]},{"name":"Life and Medical Science Application Interface Platform Development Unit, RIKEN Center for Computational Science , Kobe, Hyogo 650-0047,","place":["Japan"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,9,19]]},"reference":[{"key":"2026101714572398200_btag685-B1","doi-asserted-by":"publisher","first-page":"493","DOI":"10.1038\/s41586-024-07487-w","article-title":"Accurate structure prediction of biomolecular interactions with AlphaFold 3","volume":"630","author":"Abramson","year":"2024","journal-title":"Nature"},{"key":"2026101714572398200_btag685-B2","first-page":"460","article-title":"From mechanistic interpretability to mechanistic biology: training, evaluating, and interpreting sparse autoencoders on protein language models","volume-title":"Proceedings of the 42nd International Conference on Machine Learning, PMLR 267","author":"Adams","year":"2025"},{"key":"2026101714572398200_btag685-B3","doi-asserted-by":"publisher","author":"Akiba","year":"2019","DOI":"10.1145\/3292500.3330701"},{"key":"2026101714572398200_btag685-B4","doi-asserted-by":"publisher","first-page":"15481","DOI":"10.1038\/s41598-019-50722-y","article-title":"Structural and thermodynamic basis for the recognition of the substrate-binding cleft on hen egg lysozyme by a single-domain antibody","volume":"9","author":"Akiba","year":"2019","journal-title":"Sci Rep"},{"key":"2026101714572398200_btag685-B5","doi-asserted-by":"publisher","first-page":"661","DOI":"10.1186\/s12951-024-02900-y","article-title":"Discovery of nanobodies: a comprehensive review of their applications and potential over the past five years","volume":"22","author":"Alexander","year":"2024","journal-title":"J Nanobiotechnol"},{"key":"2026101714572398200_btag685-B6","doi-asserted-by":"publisher","first-page":"19074","DOI":"10.1038\/s41598-024-70101-6","article-title":"TEMPRO: nanobody melting temperature estimation model using protein embeddings","volume":"14","author":"Alvarez","year":"2024","journal-title":"Sci Rep"},{"key":"2026101714572398200_btag685-B7","doi-asserted-by":"publisher","DOI":"10.1101\/2023.12.12.569610","article-title":"Enhancing antibody language models with structural information","volume-title":"bioRxiv","author":"Barton","year":"2024"},{"key":"2026101714572398200_btag685-B8","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1002\/pro.3546","article-title":"Thermal stability of single-domain antibodies estimated by molecular dynamics simulations","volume":"28","author":"Bekker","year":"2019","journal-title":"Protein Sci"},{"key":"2026101714572398200_btag685-B9","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1016\/0021-9991(76)90078-4","article-title":"Efficient estimation of free energy differences from Monte Carlo data","volume":"22","author":"Bennett","year":"1976","journal-title":"J Comput Phys"},{"key":"2026101714572398200_btag685-B10","doi-asserted-by":"publisher","first-page":"3257","DOI":"10.1021\/ct300400x","article-title":"Optimization of the additive CHARMM all-atom protein force field targeting improved sampling of the backbone \u03c6, \u03c8 and side-chain \u03c71 and \u03c72 dihedral angles","volume":"8","author":"Best","year":"2012","journal-title":"J Chem Theory Comput"},{"key":"2026101714572398200_btag685-B11","doi-asserted-by":"publisher","first-page":"e82593","DOI":"10.7554\/eLife.82593","article-title":"Rapid protein stability prediction using deep learning representations","volume":"12","author":"Blaabjerg","year":"2023","journal-title":"eLife"},{"key":"2026101714572398200_btag685-B12","article-title":"Towards monosemanticity: decomposing language models with dictionary learning","author":"Bricken","year":"2023","journal-title":"Transformer Circuits Thread"},{"key":"2026101714572398200_btag685-B13","doi-asserted-by":"publisher","first-page":"e1012248","DOI":"10.1371\/journal.pcbi.1012248","article-title":"Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset","volume":"20","author":"Chu","year":"2024","journal-title":"PLoS Comput Biol"},{"key":"2026101714572398200_btag685-B14","doi-asserted-by":"publisher","first-page":"D1273","DOI":"10.1093\/nar\/gkab1021","article-title":"INDI\u2014integrated nanobody database for immunoinformatics","volume":"50","author":"Deszy\u0144ski","year":"2022","journal-title":"Nucleic Acids Res"},{"key":"2026101714572398200_btag685-B15","doi-asserted-by":"publisher","first-page":"7112","DOI":"10.1109\/TPAMI.2021.3095381","article-title":"ProtTrans: toward understanding the language of life through self-supervised learning","volume":"44","author":"Elnaggar","year":"2022","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2026101714572398200_btag685-B16","doi-asserted-by":"publisher","first-page":"8577","DOI":"10.1063\/1.470117","article-title":"A smooth particle mesh Ewald method","volume":"103","author":"Essmann","year":"1995","journal-title":"J Chem Phys"},{"key":"2026101714572398200_btag685-B17","doi-asserted-by":"publisher","first-page":"e2506316122","DOI":"10.1073\/pnas.2506316122","article-title":"Sparse autoencoders uncover biologically interpretable features in protein language model representations","volume":"122","author":"Gujral","year":"2025","journal-title":"Proc Natl Acad Sci U S A"},{"key":"2026101714572398200_btag685-B18","doi-asserted-by":"publisher","first-page":"850","DOI":"10.1126\/science.ads0018","article-title":"Simulating 500 million years of evolution with a language model","volume":"387","author":"Hayes","year":"2025","journal-title":"Science"},{"key":"2026101714572398200_btag685-B19","doi-asserted-by":"publisher","first-page":"657","DOI":"10.1006\/jmbi.2001.4662","article-title":"Yet another numbering scheme for immunoglobulin variable domains: an automatic modeling and analysis tool","volume":"309","author":"Honegger","year":"2001","journal-title":"J Mol Biol"},{"key":"2026101714572398200_btag685-B21","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/0263-7855(96)00018-5","article-title":"VMD: visual molecular dynamics","volume":"14","author":"Humphrey","year":"1996","journal-title":"J Mol Graph"},{"key":"2026101714572398200_btag685-B22","doi-asserted-by":"publisher","first-page":"20624","DOI":"10.1038\/s41598-021-98977-8","article-title":"Delicate balance among thermal stability, binding affinity, and conformational space explored by single-domain VHH antibodies","volume":"11","author":"Ikeuchi","year":"2021","journal-title":"Sci Rep"},{"key":"2026101714572398200_btag685-B23","doi-asserted-by":"publisher","first-page":"926","DOI":"10.1063\/1.445869","article-title":"Comparison of simple potential functions for simulating liquid water","volume":"79","author":"Jorgensen","year":"1983","journal-title":"J Chem Phys"},{"key":"2026101714572398200_btag685-B24","doi-asserted-by":"publisher","first-page":"362","DOI":"10.1038\/s42004-025-01771-0","article-title":"Accurate predictions of protein mutational effects accelerated with a hybrid-topology free energy protocol","volume":"8","author":"Koenekoop","year":"2025","journal-title":"Commun Chem"},{"key":"2026101714572398200_btag685-B25","doi-asserted-by":"publisher","author":"Lafita","year":"2024","DOI":"10.48550\/arXiv.2405.06729"},{"key":"2026101714572398200_btag685-B26","doi-asserted-by":"publisher","first-page":"100513","DOI":"10.1016\/j.patter.2022.100513","article-title":"Deciphering the language of antibodies using self-supervised learning","volume":"3","author":"Leem","year":"2022","journal-title":"Patterns (N Y)"},{"key":"2026101714572398200_btag685-B27","doi-asserted-by":"publisher","first-page":"5544","DOI":"10.1016\/j.csbj.2023.11.006","article-title":"DeepTM: a deep learning algorithm for prediction of melting temperature of thermophilic proteins directly from sequences","volume":"21","author":"Li","year":"2023","journal-title":"Comput Struct Biotechnol J"},{"key":"2026101714572398200_btag685-B28","doi-asserted-by":"publisher","first-page":"1123","DOI":"10.1126\/science.ade2574","article-title":"Evolutionary-scale prediction of atomic-level protein structure with a language model","volume":"379","author":"Lin","year":"2023","journal-title":"Science"},{"key":"2026101714572398200_btag685-B29","doi-asserted-by":"publisher","first-page":"2606","DOI":"10.1021\/ct300242f","article-title":"A toolkit for the analysis of free-energy perturbation calculations","volume":"8","author":"Liu","year":"2012","journal-title":"J Chem Theory Comput"},{"key":"2026101714572398200_btag685-B30","doi-asserted-by":"publisher","first-page":"5524","DOI":"10.1021\/acs.nanolett.9b02062","article-title":"Removal of a conserved disulfide bond does not compromise mechanical stability of a VHH antibody complex","volume":"19","author":"Liu","year":"2019","journal-title":"Nano Lett"},{"key":"2026101714572398200_btag685-B31","doi-asserted-by":"publisher","first-page":"2084","DOI":"10.1111\/febs.15515","article-title":"A guide to: generation and design of nanobodies","volume":"288","author":"Muyldermans","year":"2021","journal-title":"FEBS J"},{"key":"2026101714572398200_btag685-B32","doi-asserted-by":"publisher","first-page":"775","DOI":"10.1146\/annurev-biochem-063011-092449","article-title":"Nanobodies: natural single-domain antibodies","volume":"82","author":"Muyldermans","year":"2013","journal-title":"Annu Rev Biochem"},{"key":"2026101714572398200_btag685-B33","doi-asserted-by":"publisher","first-page":"1129","DOI":"10.1093\/protein\/7.9.1129","article-title":"Sequence and structure of VH domain from naturally occurring camel heavy chain immunoglobulins lacking light chains","volume":"7","author":"Muyldermans","year":"1994","journal-title":"Protein Eng"},{"key":"2026101714572398200_btag685-B34","doi-asserted-by":"publisher","first-page":"044130","DOI":"10.1063\/5.0014475","article-title":"Scalable molecular dynamics on CPU and GPU architectures with NAMD","volume":"153","author":"Phillips","year":"2020","journal-title":"J Chem Phys"},{"key":"2026101714572398200_btag685-B35","doi-asserted-by":"publisher","first-page":"1122","DOI":"10.1021\/acs.jcim.6b00162","article-title":"AlaScan: a graphical user interface for alanine scanning Free-Energy calculations","volume":"56","author":"Ramadoss","year":"2016","journal-title":"J Chem Inf Model"},{"key":"2026101714572398200_btag685-B36","author":"Rao"},{"key":"2026101714572398200_btag685-B37","doi-asserted-by":"publisher","first-page":"e2016239118","DOI":"10.1073\/pnas.2016239118","article-title":"Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences","volume":"118","author":"Rives","year":"2021","journal-title":"Proc Natl Acad Sci U S A"},{"key":"2026101714572398200_btag685-B38","doi-asserted-by":"publisher","first-page":"11905","DOI":"10.1074\/jbc.M114.614842","article-title":"Fully human VH single domains that rival the stability and cleft recognition of camelid antibodies","volume":"290","author":"Rouet","year":"2015","journal-title":"J Biol Chem"},{"key":"2026101714572398200_btag685-B39","doi-asserted-by":"publisher","first-page":"327","DOI":"10.1016\/0021-9991(77)90098-5","article-title":"Numerical integration of the Cartesian equations of motion of a system with constraints: molecular dynamics of n-alkanes","volume":"23","author":"Ryckaert","year":"1977","journal-title":"J Comput Phys"},{"key":"2026101714572398200_btag685-B40","doi-asserted-by":"publisher","first-page":"7407","DOI":"10.1038\/s41467-024-51844-2","article-title":"Fine-tuning protein language models boosts predictions across diverse tasks","volume":"15","author":"Schmirler","year":"2024","journal-title":"Nat Commun"},{"key":"2026101714572398200_btag685-B41","doi-asserted-by":"publisher","first-page":"2107","DOI":"10.1038\/s41592-025-02836-7","article-title":"InterPLM: discovering interpretable features in protein language models via sparse autoencoders","volume":"22","author":"Simon","year":"2025","journal-title":"Nat Methods"},{"key":"2026101714572398200_btag685-B42","doi-asserted-by":"publisher","first-page":"2542","DOI":"10.1038\/s41467-018-04964-5","article-title":"Clustering huge protein sequence sets in linear time","volume":"9","author":"Steinegger","year":"2018","journal-title":"Nat Commun"},{"key":"2026101714572398200_btag685-B43","doi-asserted-by":"publisher","author":"Su","DOI":"10.1038\/s41587-025-02859-7"},{"key":"2026101714572398200_btag685-B44","doi-asserted-by":"publisher","first-page":"926","DOI":"10.1093\/bioinformatics\/btu739","article-title":"UniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searches","volume":"31","author":"Suzek","year":"2015","journal-title":"Bioinformatics"},{"key":"2026101714572398200_btag685-B45","doi-asserted-by":"publisher","author":"Tsuruta","DOI":"10.52202\/079017-3688"},{"key":"2026101714572398200_btag685-B46","author":"Vig"},{"key":"2026101714572398200_btag685-B47","doi-asserted-by":"publisher","first-page":"e1012153","DOI":"10.1371\/journal.pcbi.1012153","article-title":"Supervised fine-tuning of pre-trained antibody language models improves antigen specificity prediction","volume":"21","author":"Wang","year":"2025","journal-title":"PLoS Comput Biol"},{"key":"2026101714572398200_btag685-B48","doi-asserted-by":"publisher","author":"Zhang","year":"2025","DOI":"10.48550\/arXiv.2505.02022"},{"key":"2026101714572398200_btag685-B49","doi-asserted-by":"publisher","first-page":"e2406285121","DOI":"10.1073\/pnas.2406285121","article-title":"Protein language models learn evolutionary statistics of interacting sequence motifs","volume":"121","author":"Zhang","year":"2024","journal-title":"Proc Natl Acad Sci U S A"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btag685\/71252026\/btag685.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/42\/10\/btag685\/71252026\/btag685.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/42\/10\/btag685\/71252026\/btag685.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T18:57:33Z","timestamp":1792263453000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/doi\/10.1093\/bioinformatics\/btag685\/8820484"}},"subtitle":[],"editor":[{"given":"Inanc","family":"Birol","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,9,19]]},"references-count":48,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2026,10,2]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btag685","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,10]]},"published":{"date-parts":[[2026,9,19]]},"article-number":"btag685"}}