{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T15:26:24Z","timestamp":1776093984190,"version":"3.50.1"},"reference-count":21,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2025,3,3]],"date-time":"2025-03-03T00:00:00Z","timestamp":1740960000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"AI Safety Camp"},{"name":"Future of Life Institute"},{"name":"Survival &amp; Flourishing Fund"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>Value alignment for AI is not \u201cone-size-fits-all\u201d: even polite and friendly models can still fail to represent individual user contexts and preferences, and local cultural norms. This paper presents a modular workflow for personal fine-tuning, synthesizing four core components from our previous research: (1) robust vectorization of user values and preferences, (2) a binary choice user interface (UI) approach to capturing those preferences with minimal cognitive load, (3) contrastive activation methods for steering large language models (LLMs) via difference vectors, and (4) knowledge graph integration for more auditable and structured alignment. Our approach\u2014descended from past research on \u201cTowards an End-to-End Personal Fine-Tuning Framework\u201d\u2014demonstrates how these elements can be combined to create personalized, context-rich alignment solutions. We report on user studies for the forced-choice UI, describe an experimental pipeline for deriving \u201ccontrol vectors\u201d, and propose a \u201cmoral graph\u201d method for bridging symbolic and vector-based alignment. Our findings suggest that multi-pronged personalization can significantly reduce user annotation fatigue, improve alignment fidelity, and allow for more flexible, interpretable AI behaviors.<\/jats:p>","DOI":"10.3390\/mti9030022","type":"journal-article","created":{"date-parts":[[2025,3,3]],"date-time":"2025-03-03T05:52:16Z","timestamp":1740981136000},"page":"22","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Choice Vectors: Streamlining Personal AI Alignment Through Binary Selection"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4306-7577","authenticated-orcid":false,"given":"Eleanor","family":"Watson","sequence":"first","affiliation":[{"name":"School of Computing and Engineering, University of Gloucestershire, The Park, Cheltenham GL50 2RH, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Minh","family":"Nguyen","sequence":"additional","affiliation":[{"name":"Lee Kong Chian School of Business, Singapore Management University, 81 Victoria St, Singapore 188065, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sarah","family":"Pan","sequence":"additional","affiliation":[{"name":"Electrical Engineering & Computer Science Department, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139-4307, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5699-2676","authenticated-orcid":false,"given":"Shujun","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computing and Engineering, University of Gloucestershire, The Park, Cheltenham GL50 2RH, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Watson, E., Viana, T., Sturgeon, B., Petersson, L., and Zhang, S. (2024). Towards an End-to-End Personal Fine-Tuning Framework for AI Value Alignment. Electronics, 13.","DOI":"10.3390\/electronics13204044"},{"key":"ref_2","unstructured":"Kirk, H.R., Suresh, H., Gabriel, I., Johnson, M., Hooker, S., and Prabhakaran, V. (2024). The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Goyal, N., Chang, M., and Terry, M. (2024). Designing for Human-Agent Alignment: Understanding What Humans Want from Their Agents. arXiv.","DOI":"10.1145\/3613905.3650948"},{"key":"ref_4","unstructured":"Tan, Z.-X., Carroll, M., Franklin, M., and Ashton, H. (2024). Beyond Preferences in AI Alignment. arXiv."},{"key":"ref_5","unstructured":"Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., and Ray, A. (2022). Training Language Models to Follow Instructions with Human Feedback. arXiv."},{"key":"ref_6","unstructured":"Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., and Henighan, T. (2022). Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. arXiv."},{"key":"ref_7","unstructured":"Turner, A.M., MacDiarmid, M., Udell, D., Thiergart, L., and Mini, U. (2022, December 15). Steering GPT-2-XL by Adding an Activation Vector. AI Alignment Forum. Available online: https:\/\/www.alignmentforum.org\/posts\/5spBue2z2tw4JuDCx\/steering-gpt-2-xl-by-adding-an-activation-vector."},{"key":"ref_8","unstructured":"Vogel, T. (2024, February 18). Representation Engineering Mistral-7B an Acid Trip. VGel is Me. Available online: https:\/\/vgel.me\/posts\/representation-engineering."},{"key":"ref_9","unstructured":"Panickssery, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., and Turner, A.M. (2023). Steering Llama 2 via Contrastive Activation Addition. arXiv."},{"key":"ref_10","unstructured":"Rafailov, R., Hejna, J., Sikchi, H., Finn, C., Niekum, S., Knox, W.B., and Sadigh, D. (2024). Contrastive Preference Learning: Learning from Human Feedback Without RL. arXiv."},{"key":"ref_11","unstructured":"Yuan, W., Pang, R.Y., Cho, K., Li, X., Sukhbaatar, S., Xu, J., and Weston, J. (2024). Self-Rewarding Language Models. arXiv."},{"key":"ref_12","unstructured":"Klingefjord, O., Lowe, R., and Edelman, J. (2024). What Are Human Values, and How Do We Align AI to Them?. arXiv."},{"key":"ref_13","unstructured":"Klingefjord, O., Lowe, R., and Edelman, J. (2024, January 10). The First Moral Graph: On Eliciting and Representing Values for AI Alignment. Meaning Alignment Institute. Available online: https:\/\/meaningalignment.substack.com\/p\/the-first-moral-graph."},{"key":"ref_14","unstructured":"(2023, February 15). Infinite Craft. Available online: https:\/\/neal.fun\/infinite-craft\/."},{"key":"ref_15","unstructured":"Burns, C., Ye, H., Klein, D., and Steinhardt, J. (2024). Discovering Latent Knowledge in Language Models Without Supervision. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Azaria, A., and Mitchell, T. (2023). The Internal State of an LLM Knows When It\u2019s Lying. arXiv.","DOI":"10.18653\/v1\/2023.findings-emnlp.68"},{"key":"ref_17","unstructured":"Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv."},{"key":"ref_18","unstructured":"Zou, A., Phan, L., Chen, S., Campbell, J., Guo, P., Ren, R., Pan, A., Yin, X., Mazeika, M., and Dombrowski, A.-K. (2023). Representation Engineering: A Top-Down Approach to AI Transparency. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Fourney, A., White, R.W., and Horvitz, E. (2015, January 18\u201323). Exploring Time-Dependent Concerns about Pregnancy and Childbirth from Search Logs. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, Republic of Korea.","DOI":"10.1145\/2702123.2702427"},{"key":"ref_20","unstructured":"Gao, G., Taymanov, A., Salinas, E., Mineiro, P., and Misra, D. (2024). Aligning LLM Agents by Learning Latent Preference from User Edits. arXiv."},{"key":"ref_21","unstructured":"Li, J., Zhang, Q., Yu, Y., Fu, Q., and Ye, D. (2024). More Agents Is All You Need. arXiv."}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/9\/3\/22\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:46:05Z","timestamp":1760028365000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/9\/3\/22"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,3]]},"references-count":21,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,3]]}},"alternative-id":["mti9030022"],"URL":"https:\/\/doi.org\/10.3390\/mti9030022","relation":{},"ISSN":["2414-4088"],"issn-type":[{"value":"2414-4088","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,3]]}}}