{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T11:25:14Z","timestamp":1787570714148,"version":"build-2736575974"},"reference-count":24,"publisher":"Oxford University Press (OUP)","issue":"Supplement_2","license":[{"start":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T00:00:00Z","timestamp":1787529600000},"content-version":"vor","delay-in-days":23,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Institute of General Medical Sciences of the National Institutes of Health","award":["R35GM155417"],"award-info":[{"award-number":["R35GM155417"]}]},{"DOI":"10.13039\/100023015","name":"Advanced Research Projects Agency for Health","doi-asserted-by":"publisher","award":["140D042590007"],"award-info":[{"award-number":["140D042590007"]}],"id":[{"id":"10.13039\/100023015","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,8,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>Epitope-conditioned T cell receptor (TCR) generation extends protein language modeling to the design of therapeutically relevant receptors. Reinforcement learning (RL) post-training with surrogate binding predictors can improve generation controllability, but it is vulnerable to Goodhart\u2019s Law: optimizing an imperfect surrogate reward can lead to reward inflation, distributional drift, and biologically implausible or nonspecific sequences. We investigate whether reward hacking can be mitigated through improved reward formulations without modifying the generator architecture or training pipeline and without requiring additional training data.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We introduce a plug-and-play reward-design framework for RL-based TCR generation that combines heuristic biological priors, model ensembling, and binding-specificity objectives based on max-margin and contrastive formulations. These components suppress degenerate sequences, reduce model-specific biases, and discourage cross-epitope binding. During RL fine-tuning, the proposed rewards stabilize optimization, limit surrogate-reward inflation, preserve canonical CDR3\u03b2 sequence patterns and repertoire diversity, and maintain closer alignment with experimentally validated TCR-binder distributions. In evaluations on unseen epitopes, the specificity-aware reward formulations provide the strongest overall performance, improving diversity and ground-truth distributional alignment while retaining biological authenticity and predicted target binding. These findings demonstrate that Goodhart-resistant reward design improves the reliability, controllability, and generalization of epitope-conditioned TCR generation.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability and implementation<\/jats:title>\n                    <jats:p>Code and models are available in a public repository (https:\/\/github.com\/Lee-CBG\/TCRRobustRewardDesign).<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btag499","type":"journal-article","created":{"date-parts":[[2026,7,13]],"date-time":"2026-07-13T11:27:58Z","timestamp":1783942078000},"source":"Crossref","is-referenced-by-count":0,"title":["Mitigating Goodhart\u2019s law in epitope-conditioned TCR generation using plug-and-play reward designs"],"prefix":"10.1093","volume":"42","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-5976-0673","authenticated-orcid":false,"given":"Pengfei","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Computing Augmented Intelligence, Arizona State University , Tempe, AZ 85281,","place":["United States"]},{"name":"Biodesign Institute, Arizona State University , Tempe, AZ 85281,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoyi","family":"He","sequence":"additional","affiliation":[{"name":"School of Computing Augmented Intelligence, Arizona State University , Tempe, AZ 85281,","place":["United States"]},{"name":"Biodesign Institute, Arizona State University , Tempe, AZ 85281,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fredo","family":"Guan","sequence":"additional","affiliation":[{"name":"School of Computing Augmented Intelligence, Arizona State University , Tempe, AZ 85281,","place":["United States"]},{"name":"Biodesign Institute, Arizona State University , Tempe, AZ 85281,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5740-0844","authenticated-orcid":false,"given":"Hao","family":"Mei","sequence":"additional","affiliation":[{"name":"School of Computing Augmented Intelligence, Arizona State University , Tempe, AZ 85281,","place":["United States"]},{"name":"Biodesign Institute, Arizona State University , Tempe, AZ 85281,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gloria","family":"Grama","sequence":"additional","affiliation":[{"name":"Biodesign Institute, Arizona State University , Tempe, AZ 85281,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Seojin","family":"Bang","sequence":"additional","affiliation":[{"name":"Elorian AI , CA 94301,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Heewook","family":"Lee","sequence":"additional","affiliation":[{"name":"School of Computing Augmented Intelligence, Arizona State University , Tempe, AZ 85281,","place":["United States"]},{"name":"Biodesign Institute, Arizona State University , Tempe, AZ 85281,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,8,21]]},"reference":[{"key":"2026082407020403400_btag499-B1","author":"Angermueller","year":"2019"},{"key":"2026082407020403400_btag499-B2","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1111\/imr.12772","article-title":"T cell receptor-based cancer immunotherapy: emerging efficacy and pathways of resistance","volume":"290","author":"Chandran","year":"2019","journal-title":"Immunol Rev"},{"key":"2026082407020403400_btag499-B3","first-page":"4302","article-title":"Deep reinforcement learning from human preferences","volume":"30","author":"Christiano","year":"2017","journal-title":"Adv Neural Inf Process Syst"},{"key":"2026082407020403400_btag499-B4","doi-asserted-by":"crossref","first-page":"640725","DOI":"10.3389\/fimmu.2021.640725","article-title":"TCRMatch: predicting T-cell receptor specificity based on sequence similarity to previously characterized receptors","volume":"12","author":"Chronister","year":"2021","journal-title":"Front Immunol"},{"key":"2026082407020403400_btag499-B5","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1038\/nature22383","article-title":"Quantifiable predictive features define epitope-specific T cell receptor repertoires","volume":"547","author":"Dash","year":"2017","journal-title":"Nature"},{"key":"2026082407020403400_btag499-B6","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1038\/334395a0","article-title":"T-cell antigen receptor genes and T-cell recognition","volume":"334","author":"Davis","year":"1988","journal-title":"Nature"},{"key":"2026082407020403400_btag499-B7","author":"Goodfellow","year":"2014"},{"key":"2026082407020403400_btag499-B8","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1007\/978-1-349-17295-5_4","volume-title":"Monetary Theory and Practice: The UK experience","author":"Goodhart","year":"1984"},{"key":"2026082407020403400_btag499-B9","author":"Hesslow","year":"2022"},{"key":"2026082407020403400_btag499-B10","doi-asserted-by":"crossref","first-page":"175473","DOI":"10.1109\/ACCESS.2024.3504735","article-title":"Comprehensive overview of reward engineering and shaping in advancing reinforcement learning applications","volume":"12","author":"Ibrahim","year":"2024","journal-title":"IEEE Access"},{"key":"2026082407020403400_btag499-B11","article-title":"Categorizing variants of goodhart\u2019s law","author":"Manheim","year":"2018","journal-title":"arXiv Preprint arXiv"},{"key":"2026082407020403400_btag499-B12","first-page":"2025","article-title":"A functionally validated TCR-pMHC database for TCR specificity model development","author":"Messemaker","year":"2025","journal-title":"bioRxiv"},{"key":"2026082407020403400_btag499-B13","doi-asserted-by":"crossref","first-page":"1060","DOI":"10.1038\/s42003-021-02610-3","article-title":"NetTCR-2.0 enables accurate prediction of TCR-peptide binding by using paired TCR\u03b1 and \u03b2 sequence data","volume":"4","author":"Montemurro","year":"2021","journal-title":"Commun Biol"},{"key":"2026082407020403400_btag499-B14","author":"Oord","year":"2018"},{"key":"2026082407020403400_btag499-B15","doi-asserted-by":"crossref","first-page":"27730","DOI":"10.52202\/068431-2011","article-title":"Training language models to follow instructions with human feedback","volume":"35","author":"Ouyang","year":"2022","journal-title":"Adv Neural Inf Process Syst"},{"key":"2026082407020403400_btag499-B16","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1146\/annurev.immunol.23.021704.115658","article-title":"How TCRs bind MHCs, peptides, and coreceptors","volume":"24","author":"Rudolph","year":"2006","journal-title":"Annu Rev Immunol"},{"key":"2026082407020403400_btag499-B17","article-title":"Proximal policy optimization algorithms","author":"Schulman","year":"2017","journal-title":"CoRR"},{"key":"2026082407020403400_btag499-B18","doi-asserted-by":"crossref","first-page":"9460","DOI":"10.52202\/068431-0687","article-title":"Defining and characterizing reward gaming","volume":"35","author":"Skalse","year":"2022","journal-title":"Adv Neural Inf Process Syst"},{"key":"2026082407020403400_btag499-B19","doi-asserted-by":"crossref","first-page":"1803","DOI":"10.3389\/fimmu.2020.01803","article-title":"Prediction of specific TCR-peptide binding from large dictionaries of TCR-peptide pairs","volume":"11","author":"Springer","year":"2020","journal-title":"Front Immunol"},{"key":"2026082407020403400_btag499-B20","doi-asserted-by":"publisher","DOI":"10.1101\/2021.11.18.469186","article-title":"TCR-BERT: learning the grammar of T-cell receptors for flexible antigen-binding analyses","author":"Wu","year":"2021","journal-title":"bioRxiv"},{"key":"2026082407020403400_btag499-B21","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1038\/s41592-019-0496-6","article-title":"Machine-learning-guided directed evolution for protein engineering","volume":"16","author":"Yang","year":"2019","journal-title":"Nat Methods"},{"key":"2026082407020403400_btag499-B22","doi-asserted-by":"publisher","first-page":"RP88837","DOI":"10.1101\/2023.04.12.536635","article-title":"Context-aware amino acid embedding advances analysis of TCR-epitope interactions","volume":"12","author":"Zhang","year":"2023","journal-title":"eLife"},{"key":"2026082407020403400_btag499-B23","doi-asserted-by":"crossref","first-page":"i429","DOI":"10.1093\/bioinformatics\/btaf224","article-title":"Iterative attack-and-defend framework for improving TCR-epitope binding prediction models","volume":"41","author":"Zhang","year":"2025","journal-title":"Bioinformatics"},{"key":"2026082407020403400_btag499-B24","author":"Ziegler","year":"2019"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/42\/Supplement_2\/btag499\/70726366\/btag499.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/42\/Supplement_2\/btag499\/70726366\/btag499.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T11:02:13Z","timestamp":1787569333000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/doi\/10.1093\/bioinformatics\/btag499\/8767262"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,1]]},"references-count":24,"journal-issue":{"issue":"Supplement_2","published-print":{"date-parts":[[2026,8,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btag499","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,8]]},"published":{"date-parts":[[2026,8,1]]},"article-number":"btag499"}}