{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,2]],"date-time":"2026-09-02T11:20:06Z","timestamp":1788348006533,"version":"build-2803163510"},"reference-count":51,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2026,9,2]],"date-time":"2026-09-02T00:00:00Z","timestamp":1788307200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Intell."],"abstract":"<jats:p>Large language model (LLM)-based artificial intelligence is increasingly used in ethically consequential human decision-making, yet fully autonomous machine ethics remains unrealistic, motivating architectures that support rather than replace human ethical judgment. This study introduces the Cognitive\u2013Reflective Equilibration Architecture (CREA), a cognitively grounded artificial moral advisor that operationalizes the Cognitive\u2013Reflective Equilibration Model (CREM), in which reflective reasoning guides ethical judgment from intuitive cognition toward a more advanced equilibrium among competing values, drawing on Piaget and Rawls. CREA implements CREM\u2019s 20-step process through four stage-aligned reasoning agents\u2014Cognitive, Reflective, Equilibration, and Evaluation\u2014coordinated via multi-LLM orchestration, in which auxiliary models independently explore principles, generate counterarguments, and score supporting and opposing considerations to externalize reflective deliberation. The architecture was empirically evaluated by comparing four configurations\u2014single-agent, multi-agent, multi-LLM, and multi-LLM with knowledge- and reasoning-bank augmentation\u2014across four indicators of advice quality using 500 matched execution units per configuration. All comparisons are system-internal: advice quality was scored by CREA\u2019s own multi-LLM measurement pipeline rather than by human ethicists, so the findings reflect relative differences among architectures under LLM-based self-evaluation, not normative validity. Within that scope, distributing reflective reasoning across multiple models was associated with higher reason-giving (justifiability) and normative-alignment scores relative to simpler configurations. CREA therefore offers an empirically characterized, auditable advisor architecture whose potential to scaffold human ethical judgment remains a hypothesis for user-centered validation rather than a demonstrated outcome.<\/jats:p>","DOI":"10.3390\/jintelligence14090212","type":"journal-article","created":{"date-parts":[[2026,9,2]],"date-time":"2026-09-02T10:57:23Z","timestamp":1788346643000},"page":"212","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Implementing a Cognitively Grounded Artificial Moral Advisor: A Multi-LLM Multi-Agent Approach Based on the Cognitive\u2013Reflective Equilibration Model"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-2759-582X","authenticated-orcid":false,"given":"Chulmin","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Computer Education, Sungkyunkwan University, 25-2, Seonggyungwan-ro, Jongno-gu, Seoul 03063, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"methodology","vocabulary":"credit"},{"role":"software","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"},{"role":"conceptualization","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"resources","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"given":"Seongjin","family":"Ahn","sequence":"additional","affiliation":[{"name":"Department of Computer Education, Sungkyunkwan University, 25-2, Seonggyungwan-ro, Jongno-gu, Seoul 03063, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"}]}],"member":"1968","published-online":{"date-parts":[[2026,9,2]]},"reference":[{"key":"ref_1","first-page":"225","article-title":"Varieties of artificial moral agency and the new control problem","volume":"15","author":"Arvan","year":"2022","journal-title":"Humana.Mente Journal of Philosophical Studies"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1007\/s11023-020-09525-8","article-title":"A normative approach to artificial moral agency","volume":"30","author":"Behdadi","year":"2020","journal-title":"Minds and Machines"},{"key":"ref_3","unstructured":"Belloni, A., Berger, A., Boissier, O., Bonnet, G., Bourgne, G., Chardel, P.-A., Cotton, J.-P., Evreux, N., Ganascia, J.-G., Jaillon, P., Mermet, B., Picard, G., Rever, B., Simon, G., de Swarte, T., Tessier, C., Vexler, F., Voyer, R., and Zimmermann, A. (2015). Dealing with ethical conflicts in autonomous agents and multi-agent systems. Artificial intelligence and ethics: Papers from the 2015 AAAI workshop, AAAI Press."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1007\/s11948-019-00151-x","article-title":"Artificial moral agents: A survey of the current status","volume":"26","author":"Cervantes","year":"2020","journal-title":"Science and Engineering Ethics"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chen, Y.-J., Albarqawi, A., and Chen, C.-S. (2025). Enhancing clinical decision-making: Integrating multi-agent systems with ethical AI governance. 2025 IEEE conference on computational intelligence in bioinformatics and computational biology (CIBCB), IEEE.","DOI":"10.1109\/CIBCB66090.2025.11177136"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"e2412015122","DOI":"10.1073\/pnas.2412015122","article-title":"Large language models show amplified cognitive biases in moral decision-making","volume":"122","author":"Cheung","year":"2025","journal-title":"Proceedings of the National Academy of Sciences of the United States of America"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"4831","DOI":"10.1609\/aaai.v31i1.11140","article-title":"Moral decision making frameworks for artificial intelligence","volume":"31","author":"Conitzer","year":"2017","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"256","DOI":"10.2307\/2025881","article-title":"Wide reflective equilibrium and theory acceptance in ethics","volume":"76","author":"Daniels","year":"1979","journal-title":"The Journal of Philosophy"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"100096","DOI":"10.1016\/j.chbah.2024.100096","article-title":"Making moral decisions with artificial agents as advisors. A fNIRS study","volume":"2","author":"Fabre","year":"2024","journal-title":"Computers in Human Behavior: Artificial Humans"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1007\/s13162-024-00275-9","article-title":"A theoretical framework to guide AI ethical decision making","volume":"14","author":"Ferrell","year":"2024","journal-title":"AMS Review"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1007\/s00146-024-01867-6","article-title":"What makes full artificial agents morally different","volume":"40","author":"Firt","year":"2025","journal-title":"AI & Society"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1007\/s00146-020-01089-6","article-title":"Making moral machines: Why we need artificial moral agents","volume":"36","author":"Formosa","year":"2021","journal-title":"AI & Society"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1162\/daed_a_01904","article-title":"Multi-agent systems: Technical & ethical challenges of functioning in a mixed group","volume":"151","author":"Gal","year":"2022","journal-title":"Daedalus"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"92","DOI":"10.61838\/kman.aitech.3.1.9","article-title":"Developing artificial moral agents: Key research processes, techniques, and challenges","volume":"3","author":"Ghazali","year":"2025","journal-title":"AI and Tech in Behavioral and Social Sciences"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1007\/s13347-017-0285-z","article-title":"The artificial moral advisor. The \u201cideal observer\u201d meets artificial intelligence","volume":"31","author":"Giubilini","year":"2018","journal-title":"Philosophy & Technology"},{"key":"ref_16","unstructured":"Kahneman, D. (2011). Thinking, fast and slow, Farrar, Straus and Giroux."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Kim, C., and Ahn, S. (2026). Cognitive-reflective equilibration model: An ethical decision-making framework for LLM-based AI systems. Systems, 14, (7).","DOI":"10.3390\/systems14070881"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2237","DOI":"10.1007\/s00146-024-02135-3","article-title":"Rage against the authority machines: How to design artificial moral advisors for moral enhancement","volume":"40","author":"Landes","year":"2025","journal-title":"AI & Society"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/s44336-024-00009-2","article-title":"A survey on LLM-based multi-agent systems: Workflow, infrastructure, and challenges","volume":"1","author":"Li","year":"2024","journal-title":"Vicinagearth"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"112278","DOI":"10.1016\/j.jss.2024.112278","article-title":"Agent design pattern catalogue: A collection of architectural patterns for foundation model based agents","volume":"220","author":"Liu","year":"2025","journal-title":"Journal of Systems and Software"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Liu, Y., Moore, A., Webb, J., and Vallor, S. (2022). Artificial moral advisors: A new perspective from moral psychology. Proceedings of the 2022 AAAI\/ACM conference on AI, ethics, and society, Association for Computing Machinery.","DOI":"10.1145\/3514094.3534139"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2707","DOI":"10.3390\/ai5040130","article-title":"Ethical decision-making in artificial intelligence: A logic programming approach","volume":"5","author":"Machado","year":"2024","journal-title":"AI"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1145\/3540202","article-title":"Prosocial norm emergence in multi-agent systems","volume":"17","author":"Mashayekhi","year":"2022","journal-title":"ACM Transactions on Autonomous and Adaptive Systems"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Voeneky, S., Kellmeyer, P., Mueller, O., and Burgard, W. (2022). Artificial moral agents: Conceptual issues and ethical controversy. The Cambridge handbook of responsible artificial intelligence: Interdisciplinary perspectives, Cambridge University Press.","DOI":"10.1017\/9781009207898"},{"key":"ref_25","unstructured":"Seibt, J., Hakli, R., and N\u00f8rskov, M. (2014). Artificial moral agents: Creative, autonomous, social. An approach based on evolutionary computation. Sociable robots and the future of social relations: Proceedings of robo-philosophy 2014, IOS Press."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Murukannaiah, P. K., Ajmeri, N., Jonker, C. M., and Singh, M. P. (2020). New foundations of ethical multiagent systems. Proceedings of the 19th international conference on autonomous agents and multiagent systems, International Foundation for Autonomous Agents and Multiagent Systems.","DOI":"10.65109\/BFFB6289"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"106028","DOI":"10.1016\/j.cognition.2024.106028","article-title":"People expect artificial moral advisors to be more utilitarian and distrust utilitarian moral advisors","volume":"256","author":"Myers","year":"2025","journal-title":"Cognition"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"129","DOI":"10.29173\/irie14","article-title":"Artificial moral agents: An intercultural perspective","volume":"7","author":"Nagenborg","year":"2007","journal-title":"The International Review of Information Ethics"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"322","DOI":"10.51594\/ijmer.v6i2.773","article-title":"Reviewing the ethical implications of AI in decision making processes","volume":"6","author":"Osasona","year":"2024","journal-title":"International Journal of Management & Entrepreneurship Research"},{"key":"ref_30","unstructured":"Brown, T., and Thampy, K. J. (1985). The equilibration of cognitive structures: The central problem of intellectual development, University of Chicago Press. (Original work published 1975)."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"128404","DOI":"10.1016\/j.eswa.2025.128404","article-title":"AgentAI: A comprehensive survey on autonomous agents in distributed AI for Industry 4.0","volume":"291","author":"Piccialli","year":"2025","journal-title":"Expert Systems with Applications"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"e213","DOI":"10.1017\/S0033291725101311","article-title":"Ethical decision-making for AI in mental health: The Integrated Ethical Approach for Computational Psychiatry (IEACP) framework","volume":"55","author":"Putica","year":"2025","journal-title":"Psychological Medicine"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1007\/s13347-025-00864-x","article-title":"Can AI rely on the systematicity of truth? The challenge of modelling normative domains","volume":"38","author":"Queloz","year":"2025a","journal-title":"Philosophy & Technology"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1007\/s13347-025-00896-3","article-title":"On the fundamental limitations of AI moral advisors","volume":"38","author":"Queloz","year":"2025b","journal-title":"Philosophy & Technology"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"May, L., and Delston, J. B. (2017). A theory of justice. Applied ethics: A multicultural approach, Routledge. [6th ed.].","DOI":"10.4324\/9781315097176"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1571","DOI":"10.1016\/j.dss.2006.03.003","article-title":"Decision support for ethical problem solving: A multi-agent approach","volume":"43","author":"Robbins","year":"2007","journal-title":"Decision Support Systems"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"ooaf076","DOI":"10.1093\/jamiaopen\/ooaf076","article-title":"A community-based approach to ethical decision-making in artificial intelligence for health care","volume":"8","author":"Senghor","year":"2025","journal-title":"JAMIA Open"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"855","DOI":"10.1007\/s00146-022-01569-x","article-title":"Artificial virtuous agents in a multi-agent tragedy of the commons","volume":"39","author":"Stenseke","year":"2024","journal-title":"AI & Society"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1145\/3717413.3717424","article-title":"Enabling moral agency in distributed energy management: An ethics score for negotiations in multi-agent systems","volume":"4","author":"Stomberg","year":"2024","journal-title":"ACM Sigenergy Energy Informatics Review"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Tassella, M., Chaput, R., and Guillermin, M. (2023). Artificial moral advisors: Enhancing human ethical decision-making. 2023 IEEE international symposium on ethics in engineering, science, and technology (ETHICS), IEEE.","DOI":"10.1109\/ETHICS57328.2023.10155026"},{"key":"ref_41","first-page":"132","article-title":"Implementations in machine ethics: A survey","volume":"53","author":"Tolmeijer","year":"2020","journal-title":"ACM Computing Surveys"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"454","DOI":"10.1111\/j.1756-8765.2010.01095.x","article-title":"A conceptual and computational model of moral decision making in human and artificial agents","volume":"2","author":"Wallach","year":"2010","journal-title":"Topics in Cognitive Science"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., Zhao, W. X., Wei, Z., and Wen, J. (2024). A survey on large language model based autonomous agents. Frontiers of Computer Science, 18, (6).","DOI":"10.1007\/s11704-024-40231-1"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1186\/s42492-020-00063-9","article-title":"Modeling of moral decisions with deep learning","volume":"3","author":"Wiedeman","year":"2020","journal-title":"Visual Computing for Industry, Biomedicine, and Art"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Woodgate, J. (2025). Ethical decision-making in multi-agent systems. Proceedings of the 24th international conference on autonomous agents and multiagent systems (AAMAS 2025), International Foundation for Autonomous Agents and Multiagent Systems.","DOI":"10.65109\/PFQO3928"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"121101","DOI":"10.1007\/s11432-024-4222-0","article-title":"The rise and potential of large language model based agents: A survey","volume":"68","author":"Xi","year":"2025","journal-title":"Science China Information Sciences"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Yamani, A., Baslyman, M., and Ahmed, M. (2025). Multi-agent LLMs as ethics advocates for AI-based systems. Proceedings of the 2025 IEEE 33rd international requirements engineering conference workshops (REW), IEEE.","DOI":"10.1109\/REW66121.2025.00078"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.cogsys.2017.02.005","article-title":"Computational models of ethical decision-making: A coherence-driven reflective equilibrium model","volume":"46","author":"Yilmaz","year":"2017","journal-title":"Cognitive Systems Research"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"2605","DOI":"10.1007\/s43681-024-00566-8","article-title":"Normativity and AI moral agency","volume":"5","author":"Zafar","year":"2025","journal-title":"AI and Ethics"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"46595","DOI":"10.52202\/075280-2020","article-title":"Judging LLM-as-a-judge with MT-bench and chatbot arena","volume":"36","author":"Zheng","year":"2023","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Liu, L., Yu, J., and Zhang, D. (2026). LLM-based multi-agent orchestration: A survey of frameworks, communication protocols, and emerging patterns. Future Inernet, 18, (6).","DOI":"10.3390\/fi18060326"}],"container-title":["Journal of Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-3200\/14\/9\/212\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,2]],"date-time":"2026-09-02T11:11:24Z","timestamp":1788347484000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-3200\/14\/9\/212"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,2]]},"references-count":51,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["jintelligence14090212"],"URL":"https:\/\/doi.org\/10.3390\/jintelligence14090212","relation":{},"ISSN":["2079-3200"],"issn-type":[{"value":"2079-3200","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,2]]}}}