{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,16]],"date-time":"2026-10-16T17:55:39Z","timestamp":1792173339082,"version":"4.4.0"},"reference-count":61,"publisher":"National Academy of Sciences","issue":"42","license":[{"start":{"date-parts":[[2026,10,16]],"date-time":"2026-10-16T00:00:00Z","timestamp":1792108800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/100023581","name":"NSF | National Science Foundation Graduate Research Fellowship Program","doi-asserted-by":"publisher","award":["DGE-214675"],"award-info":[{"award-number":["DGE-214675"]}],"id":[{"id":"10.13039\/100023581","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000893","name":"Simons Foundation","doi-asserted-by":"publisher","award":["SFI-MPS-TAF-0008529-14"],"award-info":[{"award-number":["SFI-MPS-TAF-0008529-14"]}],"id":[{"id":"10.13039\/100000893","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.pnas.org"],"crossmark-restriction":true},"short-container-title":["Proc. Natl. Acad. Sci. U.S.A."],"published-print":{"date-parts":[[2026,10,20]]},"abstract":"<jats:p>Activation steering, or direct manipulation of internal activations to guide large language model (LLM) responses toward specific semantic concepts, is emerging as a promising avenue for both understanding how semantic concepts are stored within LLMs and advancing LLM capabilities. Yet, existing steering methods are remarkably brittle, with seemingly nonsteerable concepts becoming completely steerable based on subtle algorithmic choices in how concept-related features are extracted. In this work, we introduce an attention-guided framework for activation steering that overcomes three core challenges associated with steering: 1) automatic selection of relevant token embeddings for extracting concept-related features; 2) accounting for heterogeneity of concept-related features across LLM activations; and 3) identification of layers most relevant for steering. Across a steering benchmark of 512 semantic concepts, our framework substantially improved steering over previous state-of-the-art (nearly doubling the number of successfully steered concepts) across model architectures and sizes (up to 70 billion parameter models). Furthermore, we use our framework to shed light on the distribution of concept-specific features across LLM layers. Overall, our framework opens further avenues for developing efficient, highly scalable fine-tuning algorithms for industry-scale LLMs.<\/jats:p>","DOI":"10.1073\/pnas.2609978123","type":"journal-article","created":{"date-parts":[[2026,10,16]],"date-time":"2026-10-16T17:39:04Z","timestamp":1792172344000},"update-policy":"https:\/\/doi.org\/10.1073\/pnas.cm10313","source":"Crossref","is-referenced-by-count":0,"title":["Efficient and accurate steering of large language models through attention-guided feature learning"],"prefix":"10.1073","volume":"123","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-7132-403X","authenticated-orcid":false,"given":"Parmida","family":"Davarmanesh","sequence":"first","affiliation":[{"id":[{"id":"https:\/\/ror.org\/042nb2s44","id-type":"ROR","asserted-by":"publisher"}],"name":"Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology","place":["Cambridge"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7072-2912","authenticated-orcid":false,"given":"Ashia","family":"Wilson","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/042nb2s44","id-type":"ROR","asserted-by":"publisher"}],"name":"Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology","place":["Cambridge"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0878-815X","authenticated-orcid":false,"given":"Adityanarayanan","family":"Radhakrishnan","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/042nb2s44","id-type":"ROR","asserted-by":"publisher"}],"name":"Mathematics, Massachusetts Institute of Technology","place":["Cambridge"]},{"id":[{"id":"https:\/\/ror.org\/042nb2s44","id-type":"ROR","asserted-by":"publisher"}],"name":"Broad Institute of Massachusetts Institute of Technology and Harvard","place":["Cambridge"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"341","published-online":{"date-parts":[[2026,10,16]]},"reference":[{"key":"e_1_3_4_1_2","unstructured":"OpenAI GPT-4o system card. arXiv [Preprint] (2024). https:\/\/arxiv.org\/abs\/2410.21276 (Accessed 08 April 2026)."},{"key":"e_1_3_4_2_2","unstructured":"Anthropic \u201cThe Claude 3 model family: Opus sonnet haiku\u201d (Technical report Anthropic 2024)."},{"key":"e_1_3_4_3_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aea6792"},{"key":"e_1_3_4_4_2","unstructured":"A. Zou et al. Representation engineering: A top-down approach to AI transparency. arXiv [Preprint] (2023). https:\/\/arxiv.org\/abs\/2310.01405 (Accessed 08 April 2026)."},{"key":"e_1_3_4_5_2","unstructured":"Z. Wu et al. \u201cSteering LLMs? Even simple baselines outperform sparse autoencoders\u201d in International Conference on Machine Learning (2025)."},{"key":"e_1_3_4_6_2","unstructured":"J. Kim J. Evans A. Schein Linear representations of political perspective emerge in large language models. arXiv [Preprint] (2025). https:\/\/arxiv.org\/abs\/2503.02080 (Accessed 08 April 2026)."},{"key":"e_1_3_4_7_2","doi-asserted-by":"crossref","unstructured":"A. Arditi et al. \u201cRefusal in language models is mediated by a single direction\u201d in Advances in Neural Information Processing Systems (2024).","DOI":"10.52202\/079017-4322"},{"key":"e_1_3_4_8_2","doi-asserted-by":"crossref","unstructured":"N. Nanda A. Lee M. Wattenberg \u201cEmergent linear representations in world models of self-supervised sequence models\u201d in Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP Y. Belinkov et al. Eds. (Association for Computational Linguistics Singapore 2023) pp. 16\u201330.","DOI":"10.18653\/v1\/2023.blackboxnlp-1.2"},{"key":"e_1_3_4_9_2","unstructured":"Y. Jiang G. Rajendran P. K. Ravikumar B. Aragam V. Veitch \u201cOn the origins of linear representations in large language models\u201d in International Conference on Machine Learning (2024)."},{"key":"e_1_3_4_10_2","unstructured":"S. Marks M. Tegmark \u201cThe geometry of truth: Emergent linear structure in large language model representations of true\/false datasets\u201d in Conference on Language Modeling (2024)."},{"key":"e_1_3_4_11_2","unstructured":"K. Park Y. J. Choe V. Veitch \u201cThe linear representation hypothesis and the geometry of large language models\u201d in Workshop on Causal Representation Learning at Advances in Neural Information Processing Systems (2023)."},{"key":"e_1_3_4_12_2","doi-asserted-by":"crossref","unstructured":"C. Tigges O. J. Hollinsworth A. Geiger N. Nanda \u201cLanguage models linearly represent sentiment\u201d in Workshop on Mechanistic Interpretability at International Conference on Machine Learning (2024).","DOI":"10.18653\/v1\/2024.blackboxnlp-1.5"},{"key":"e_1_3_4_13_2","unstructured":"J. Yu X. Lin Z. Yu X. Xing \u201cLLM-Fuzzer: Scaling assessment of large language model jailbreaks\u201d in USENIX Security Symposium (2024)."},{"key":"e_1_3_4_14_2","doi-asserted-by":"crossref","unstructured":"A. Mehrotra et al. \u201cTree of attacks: Jailbreaking black-box LLMs automatically\u201d in Advances in Neural Information Processing Systems (2024).","DOI":"10.52202\/079017-1952"},{"key":"e_1_3_4_15_2","unstructured":"Z. Wei Y. Liu N. B. Erichson \u201cEmoji attack: Enhancing jailbreak attacks against judge LLM detection\u201d in International Conference on Machine Learning (2025)."},{"key":"e_1_3_4_16_2","doi-asserted-by":"crossref","unstructured":"P. Chao et al. \u201cJailbreaking black box large language models in twenty queries\u201d in IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) (2025).","DOI":"10.1109\/SaTML64287.2025.00010"},{"key":"e_1_3_4_17_2","unstructured":"A. Zou et al. Universal and transferable adversarial attacks on aligned language models. arXiv [Preprint] (2023). https:\/\/arxiv.org\/abs\/2307.15043 (Accessed 29 March 2026)."},{"key":"e_1_3_4_18_2","unstructured":"S. Wu G. Jin W. Huang J. Wang X. Huang \u201cActivation steering meets preference optimization\u201d in Asian Conference on Machine Learning (2025)."},{"key":"e_1_3_4_19_2","doi-asserted-by":"crossref","unstructured":"Q. Li et al. \u201cInternal activation revision: Safeguarding vision language models without parameter update\u201d in Conference on Artificial Intelligence (2025).","DOI":"10.1609\/aaai.v39i26.34954"},{"key":"e_1_3_4_20_2","unstructured":"J. Park M. Seo J. Choi Risk-adaptive activation steering for safe multimodal large language models. arXiv [Preprint] (2025). https:\/\/arxiv.org\/abs\/2510.13698 (Accessed 29 March 2026)."},{"key":"e_1_3_4_21_2","unstructured":"S. Liu H. Ye J. Zou \u201cReducing hallucinations in large vision-language models via latent space steering\u201d in International Conference on Learning Representations (2025)."},{"key":"e_1_3_4_22_2","doi-asserted-by":"crossref","unstructured":"K. Konen et al. \u201cStyle vectors for steering generative large language models\u201d in Findings of the Association for Computational Linguistics: EACL 2024 (2024) pp. 782\u2013802.","DOI":"10.18653\/v1\/2024.findings-eacl.52"},{"key":"e_1_3_4_23_2","unstructured":"A. M. Turner et al. Steering language models with activation engineering. arXiv [Preprint] (2023). https:\/\/arxiv.org\/abs\/2308.10248 (Accessed 29 March 2026)."},{"key":"e_1_3_4_24_2","doi-asserted-by":"crossref","unstructured":"N. Subramani N. Suresh M Peters. \u201cExtracting latent steering vectors from pretrained language models\u201d in Findings of the Association for Computational Linguistics: ACL 2022 (2022) pp. 566\u2013581.","DOI":"10.18653\/v1\/2022.findings-acl.48"},{"key":"e_1_3_4_25_2","unstructured":"E. Hernandez B. Z. Li J. Andreas \u201cInspecting and editing knowledge representations in language models\u201d in Conference on Language Modeling (2024)."},{"key":"e_1_3_4_26_2","doi-asserted-by":"crossref","unstructured":"K. Li O. Patel F. Vi\u00e9gas H. Pfister M. Wattenberg \u201cInference-time intervention: Eliciting truthful answers from a language model\u201d in Workshop on Causal Representation Learning at Advances in Neural Information Processing Systems (2023).","DOI":"10.52202\/075280-1797"},{"key":"e_1_3_4_27_2","doi-asserted-by":"crossref","unstructured":"Y. Cao et al. \u201cPersonalized steering of large language models: Versatile steering vectors through bi-directional preference optimization\u201d in Advances in Neural Information Processing Systems (2024).","DOI":"10.52202\/079017-1567"},{"key":"e_1_3_4_28_2","unstructured":"E. Todd et al. \u201cFunction vectors in large language models\u201d in International Conference on Learning Representations (2024)."},{"key":"e_1_3_4_29_2","unstructured":"R. Chen A. Arditi H. Sleight O. Evans J. Lindsey Persona vectors: Monitoring and controlling character traits in language models. arXiv [Preprint] (2025). https:\/\/arxiv.org\/abs\/2507.21509 (Accessed 29 March 2026)."},{"key":"e_1_3_4_30_2","doi-asserted-by":"crossref","unstructured":"N. Rimsky et al. \u201cSteering llama 2 via contrastive activation addition\u201d in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics L. W. Ku A. Martins V. Srikumar Eds. (2024) pp. 15504\u201315522.","DOI":"10.18653\/v1\/2024.acl-long.828"},{"key":"e_1_3_4_31_2","unstructured":"S. Dathathri et al. \u201cPlug and play language models: A simple approach to controlled text generation\u201d in International Conference on Learning Representations (2020)."},{"key":"e_1_3_4_32_2","doi-asserted-by":"crossref","unstructured":"T. Wang \u201cAdaptive activation steering: A tuning-free LLM truthfulness improvement method for diverse hallucinations categories in Proceedings of the ACM on Web Conference 2025 (2025) pp. 2562\u20132578.","DOI":"10.1145\/3696410.3714640"},{"key":"e_1_3_4_33_2","doi-asserted-by":"crossref","unstructured":"D. Gottesman M. Geva \u201cEstimating knowledge in large language models without generating a single token\u201d in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing Y. Al-Onaizan M. Bansal Y. N. Chen Eds. (2024) pp. 3994\u20134019.","DOI":"10.18653\/v1\/2024.emnlp-main.232"},{"key":"e_1_3_4_34_2","unstructured":"C. Burns H. Ye D. Klein J. Steinhardt Discovering latent knowledge in language models without supervision. arXiv [Preprint] (2022). https:\/\/arxiv.org\/abs\/2212.03827 (Accessed 29 March 2026)."},{"key":"e_1_3_4_35_2","unstructured":"L. Yu V. Do K. Hambardzumyan N. Cancedda \u201cRobust LLM safeguarding via refusal feature adversarial training\u201d in International Conference on Learning Representations (2025)."},{"key":"e_1_3_4_36_2","doi-asserted-by":"crossref","unstructured":"J. Zhao J. Huang Z. Wu D. Bau W. Shi \u201cLLMs encode harmfulness and refusal separately\u201d in Advances in Neural Information Processing Systems (2025).","DOI":"10.52202\/085713-4688"},{"key":"e_1_3_4_37_2","unstructured":"Q. Zhang et al. \u201cTell your model where to attend: Post-hoc attention steering for LLMs\u201d in International Conference on Learning Representations (2024)."},{"key":"e_1_3_4_38_2","unstructured":"A Vaswani et al. \u201cAttention is all you need\u201d in Advances in Neural Information Processing Systems (2017)."},{"key":"e_1_3_4_39_2","unstructured":"D. Hendrycks K. Gimpel Gaussian error linear units (gelus). ArXiv [Preprint] (2016). https:\/\/doi.org\/10.48550\/arXiv.1606.08415 (Accessed 29 March 2026)."},{"key":"e_1_3_4_40_2","unstructured":"J. L. Ba J. R. Kiros G. E. Hinton Layer normalization. arXiv [Preprint] (2016). https:\/\/arxiv.org\/abs\/1607.06450 (Accessed 08 April 2026)."},{"key":"e_1_3_4_41_2","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava N.","year":"2014","unstructured":"N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929\u20131958 (2014).","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_4_42_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.adi5639"},{"key":"e_1_3_4_43_2","unstructured":"S. Liu H. Ye L. Xing J. Zou In-context vectors: Making in context learning more effective and controllable through latent space steering. arXiv [Preprint] (2023). https:\/\/arxiv.org\/abs\/2311.06668 (Accessed 29 March 2026)."},{"key":"e_1_3_4_44_2","doi-asserted-by":"crossref","unstructured":"A. Stolfo V. Balachandran S. Yousefi E. Horvitz B. Nushi \u201cImproving instruction-following in language models through activation steering\u201d in International Conference on Learning Representations (2025).","DOI":"10.32388\/BTHT4K"},{"key":"e_1_3_4_45_2","doi-asserted-by":"crossref","unstructured":"A. Zou et al. \u201cImproving alignment and robustness with circuit breakers\u201d in Advances in Neural Information Processing Systems (2024).","DOI":"10.52202\/079017-2651"},{"key":"e_1_3_4_46_2","unstructured":"T. van der Weij M. Poesio N. Schoots Extending activation steering to broad skills and multiple behaviours. arXiv [Preprint] (2024). https:\/\/arxiv.org\/abs\/2403.05767 (Accessed 29 March 2026)."},{"key":"e_1_3_4_47_2","unstructured":"AI@Meta The Llama 3 herd of models. arXiv [Preprint] (2024). https:\/\/arxiv.org\/abs\/2407.21783."},{"key":"e_1_3_4_48_2","unstructured":"A. Yang Qwen3 technical report. arXiv [Preprint] (2025). https:\/\/arxiv.org\/abs\/2505.09388."},{"key":"e_1_3_4_49_2","unstructured":"Gemma Team Gemma 2: Improving open language models at a practical size. arXiv [Preprint] (2024). https:\/\/arxiv.org\/abs\/2408.00118 (Accessed 12 May 2026)."},{"key":"e_1_3_4_50_2","unstructured":"L. Tunstall \u201cZephyr: Direct distillation of LM alignment\u201d in First Conference on Language Modeling (COLM 2024)."},{"key":"e_1_3_4_51_2","doi-asserted-by":"crossref","unstructured":"P. Venkateswaran D. Contractor \u201cSpotlight your instructions: Instruction-following with dynamic attention steering\u201d in Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) (2026) pp. 3752\u20133770.","DOI":"10.18653\/v1\/2026.eacl-long.174"},{"key":"e_1_3_4_52_2","unstructured":"V. Guardieiro A. Stein A. Khare E. Wong Instruction following by boosting attention of large language models. arXiv [Preprint] (2025). https:\/\/arxiv.org\/abs\/2506.13734 (Accessed 08 April 2026)."},{"key":"e_1_3_4_53_2","unstructured":"A. Zou et al. Universal and transferable adversarial attacks on aligned language models. arXiv [Preprint] (2023). https:\/\/arxiv.org\/abs\/2307.15043 (Accessed 29 March 2026)."},{"key":"e_1_3_4_54_2","doi-asserted-by":"crossref","unstructured":"P. Chao et al. \u201cJailbreakBench: An open robustness benchmark for jailbreaking large language models\u201d in Neural Information Processing Systems Datasets and Benchmarks Track (2024).","DOI":"10.52202\/079017-1745"},{"key":"e_1_3_4_55_2","unstructured":"HackerRank HackerRank coding challenges (2025). https:\/\/www.hackerrank.com\/. Accessed 02 May 2025."},{"key":"e_1_3_4_56_2","doi-asserted-by":"crossref","unstructured":"J. Li X. Cheng X. Zhao J. Y. Nie J. R. Wen \u201cHaluEval: A large-scale hallucination evaluation benchmark for large language models\u201d in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing H. Bouamor J. Pino K. Bali Eds. (Association for Computational Linguistics Singapore 2023) pp. 6449\u20136464.","DOI":"10.18653\/v1\/2023.emnlp-main.397"},{"key":"e_1_3_4_57_2","unstructured":"A. Mishra et al. Fine-grained hallucination detection and editing for language models. arXiv [Preprint] (2024). https:\/\/arxiv.org\/abs\/2401.06855 (Accessed 29 March 2026)."},{"key":"e_1_3_4_58_2","doi-asserted-by":"crossref","unstructured":"S. Chen et al. \u201cFELM: Benchmarking factuality evaluation of large language models\u201d in Thirty-Seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track (2023).","DOI":"10.52202\/075280-1927"},{"key":"e_1_3_4_59_2","unstructured":"G. G\u00f3ral M. Winkels S. Basart Depth-wise activation steering for honest language models. arXiv [Preprint] (2025). https:\/\/arxiv.org\/abs\/2512.07667 (Accessed 29 March 2026)."},{"key":"e_1_3_4_60_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-021-03819-2"},{"key":"e_1_3_4_61_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-025-10014-0"}],"container-title":["Proceedings of the National Academy of Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pnas.org\/doi\/pdf\/10.1073\/pnas.2609978123","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,16]],"date-time":"2026-10-16T17:39:37Z","timestamp":1792172377000},"score":1,"resource":{"primary":{"URL":"https:\/\/pnas.org\/doi\/10.1073\/pnas.2609978123"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,16]]},"references-count":61,"journal-issue":{"issue":"42","published-print":{"date-parts":[[2026,10,20]]}},"alternative-id":["10.1073\/pnas.2609978123"],"URL":"https:\/\/doi.org\/10.1073\/pnas.2609978123","relation":{},"ISSN":["0027-8424","1091-6490"],"issn-type":[{"value":"0027-8424","type":"print"},{"value":"1091-6490","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,16]]},"assertion":[{"value":"2026-03-19","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-10-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2609978123"}}