{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,14]],"date-time":"2026-02-14T00:48:51Z","timestamp":1771030131014,"version":"3.50.1"},"reference-count":54,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2023,9,28]],"date-time":"2023-09-28T00:00:00Z","timestamp":1695859200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"U.S. Army","award":["W15QKN-18-D-0040"],"award-info":[{"award-number":["W15QKN-18-D-0040"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>Many systems engineering projects begin with the involvement of stakeholders to aid in decision-making processes. As an application of systems engineering, systems architecture involves the documentation of stakeholder needs gathered via elicitation and the transformation of these needs into requirements for a system. Within human\u2013machine teaming, systems architecture allows for the creation of a system with desired characteristics elicited from stakeholders involved with the project or system. Though stakeholders can be excellent sources for expert opinion, vested interests in a project may potentially bias stakeholders and impact decision-making processes. These biases may influence the design of the system architecture, potentially resulting in a system that is developed with unbalanced and misrepresented stakeholder preferences. This paper presents an activity analysis of the Stakeholder Needs and Requirements Process as described in the Systems Engineering Body of Knowledge (SEBoK) to identify potential biases associated with this elicitation process. As part of the research presented in this paper, a workshop was conducted where currently practicing systems architects provided feedback regarding perceptions of biases encountered during the elicitation process. The findings of this research will aid systems architects, developers, and users in understanding how biases may impact stakeholder elicitation within the architecting process.<\/jats:p>","DOI":"10.3390\/systems11100499","type":"journal-article","created":{"date-parts":[[2023,9,29]],"date-time":"2023-09-29T05:48:13Z","timestamp":1695966493000},"page":"499","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Biases in Stakeholder Elicitation as a Precursor to the Systems Architecting Process"],"prefix":"10.3390","volume":"11","author":[{"given":"Taylor","family":"Yeazitzis","sequence":"first","affiliation":[{"name":"Department of Psychology, The University of Alabama in Huntsville, Huntsville, AL 35899, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kristin","family":"Weger","sequence":"additional","affiliation":[{"name":"Department of Psychology, The University of Alabama in Huntsville, Huntsville, AL 35899, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bryan","family":"Mesmer","sequence":"additional","affiliation":[{"name":"Department of Industrial & Systems Engineering and Engineering Management, The University of Alabama in Huntsville, Huntsville, AL 35899, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joseph","family":"Clerkin","sequence":"additional","affiliation":[{"name":"Department of Industrial & Systems Engineering and Engineering Management, The University of Alabama in Huntsville, Huntsville, AL 35899, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9910-371X","authenticated-orcid":false,"given":"Douglas","family":"Van Bossuyt","sequence":"additional","affiliation":[{"name":"Naval Postgraduate School, Monterey, CA 93943, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Madni, A.M., and Madni, C.C. (2018). Architectural framework for exploring adaptive human-machine teaming options in simulated dynamic environments. Systems, 6.","DOI":"10.3390\/systems6040044"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"724587","DOI":"10.1155\/2012\/724587","article-title":"Estimation algorithm of machine operational intention by Bayes filtering with self-organizing map","volume":"2012","author":"Suzuki","year":"2012","journal-title":"Adv. Hum.-Comput. Interact."},{"key":"ref_3","unstructured":"Cloutier, R.J. (2022). The Guide to the Systems Engineering Body of Knowledge (SEBoK), The Trustees of the Stevens Institute of Technology. v. 2.7."},{"key":"ref_4","unstructured":"Kahneman, D. (2011). Thinking, Fast and Slow, Macmillan."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Payne, J.W., Bettman, J.R., and Johnson, E.J. (1993). The Adaptive Decision Maker, Cambridge University Press.","DOI":"10.1017\/CBO9781139173933"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"B1","DOI":"10.1016\/S0010-0277(03)00141-0","article-title":"Not so fast! (and not so frugal!): Rethinking the recognition heuristic","volume":"90","author":"Oppenheimer","year":"2003","journal-title":"Cognition"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ellis, G. (2018). Cognitive Biases in Visualizations, Springer.","DOI":"10.1007\/978-3-319-95831-6"},{"key":"ref_8","unstructured":"Freeman, R.E. (1984). Strategic Management: A Stakeholder Approach, Pitman."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"92","DOI":"10.2307\/258888","article-title":"A stakeholder framework for analyzing and evaluating corporate social performance","volume":"20","author":"Clarkson","year":"1995","journal-title":"Acad. Manag. Rev."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"853","DOI":"10.2307\/259247","article-title":"Toward a Theory of Stakeholder Identification and Salience: Defining the Principle of Who and What Really Counts","volume":"22","author":"Mitchell","year":"1997","journal-title":"Acad. Manag. Rev."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Babar, M.A., Zhu, L., and Jeffery, R. (2004, January 13\u201316). A framework for classifying and comparing software architecture evaluation methods. Proceedings of the 2004 Australian Software Engineering Conference, Melbourne, VIC, Australia.","DOI":"10.1109\/ASWEC.2004.1290484"},{"key":"ref_12","first-page":"85","article-title":"Project planning and control: Early engagement of project stakeholders","volume":"2","author":"Caron","year":"2014","journal-title":"J. Mod. Proj. Manag."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Burgman, M.A. (2004). Expert Frailities in Conservation Risk Assessment and Listing Decisions, Royal Zoological Society of New South Wales.","DOI":"10.7882\/FS.2004.054"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1111\/2041-210X.12857","article-title":"A practical guide to structured expert elicitation using the IDEA protocol","volume":"9","author":"Hemming","year":"2018","journal-title":"Methods Ecol. Evol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1016\/S0377-2217(01)00113-8","article-title":"Performance-based assessment of expertise: How to decide if someone is an expert or not","volume":"136","author":"Shanteau","year":"2002","journal-title":"Eur. J. Oper. Res."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"348","DOI":"10.1037\/0096-3445.115.4.348","article-title":"Measuring the vague meanings of probability terms","volume":"115","author":"Wallsten","year":"1986","journal-title":"J. Exp. Psychol. Gen."},{"key":"ref_17","unstructured":"Janis, I. (1991). A First Look at Communication Theory, McGrawHill."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Rougier, J., Sparks, S., and Hill, L. (2013). Risk and Uncertainty Assessment for Natural Hazards, Cambridge University Press.","DOI":"10.1017\/CBO9781139047562"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"9020","DOI":"10.1073\/pnas.1008636108","article-title":"How social influence can undermine the wisdom of crowd effect","volume":"108","author":"Lorenz","year":"2011","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_20","first-page":"84","article-title":"The Effect of Overconfidence and Optimism Bias on Stock Investment Decisions with Financial Literature as Moderating Variable","volume":"12","author":"Arik","year":"2021","journal-title":"Eurasia Econ. Bus."},{"key":"ref_21","first-page":"93","article-title":"Appraising the Manifestation of Optimism Bias and Its Impact on Human Perception of Cyber Security: A Meta Analysis","volume":"14","author":"Alnifie","year":"2023","journal-title":"J. Inf. Secur."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhang, Z., and Li, H. (2019, January 24\u201325). Research on Human Machine Viewpoint of Architecture for C4ISR System. Proceedings of the 2019 11th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), Hangzhou, China.","DOI":"10.1109\/IHMSC.2019.00028"},{"key":"ref_23","unstructured":"Chatzipanos, P.A., and Giotis, T. (2014). Cognitive Biases as Project & Program Complexity Enhancers: The Astypalea Project, Project Management Institute."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1124","DOI":"10.1126\/science.185.4157.1124","article-title":"Judgment under Uncertainty: Heuristics and Biases","volume":"185","author":"Tversky","year":"1974","journal-title":"Science"},{"key":"ref_25","first-page":"26","article-title":"Effect of Representativeness Bias on Investment Decision Making","volume":"5","author":"Irshad","year":"2016","journal-title":"Manag. Adm. Sci. Rev."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1093\/intqhc\/mzu083","article-title":"Systematic biases in group decision-making: Implications for patient safety","volume":"26","author":"Mannion","year":"2014","journal-title":"Int. J. Qual. Health Care"},{"key":"ref_27","unstructured":"Gulati, A., Lozano, M.A., Lepri, B., and Oliver, N. (2022). BIASeD: Bringing Irrationality into Automated System Design. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1177\/87569728211049046","article-title":"Top Ten Behavioral Biases in Project Management: An Overview","volume":"52","author":"Flyvbjerg","year":"2021","journal-title":"Proj. Manag. J."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"956","DOI":"10.1287\/isre.2013.0497","article-title":"Do recommender systems manipulate consumer preferences? A study of anchoring effects","volume":"24","author":"Adomavicius","year":"2013","journal-title":"Inf. Syst. Res."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Souza, P.E.U., Carvalho Chanel, C.P., Dehais, F., and Givigi, S. (2016, January 9\u201312). Towards human-robot interaction: A framing effect experiment. Proceedings of the 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Budapest, Hungary,.","DOI":"10.1109\/SMC.2016.7844521"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"790","DOI":"10.1080\/10447318.2022.2049134","article-title":"Communicating the limitations of AI: The effect of message framing and ownership on trust in artificial intelligence","volume":"39","author":"Kim","year":"2023","journal-title":"Int. J. Hum. Comput. Interact."},{"key":"ref_32","first-page":"760","article-title":"Quality control and due diligence in project management: Getting decisions right by taking the outside view","volume":"3","author":"Flyvbjerg","year":"2012","journal-title":"Int. J. Proj. Manag."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.eiar.2017.10.003","article-title":"Stakeholders\u2019 manipulation of Environmental Impact Assessment","volume":"68","year":"2018","journal-title":"Environ. Impact Assess. Rev."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1016\/S0166-4115(08)61332-1","article-title":"Biases in categorization","volume":"68","year":"1990","journal-title":"Adv. Psychol."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1108\/13632540610664698","article-title":"Unpacking the halo effect: Reputation and crisis management","volume":"10","author":"Coombs","year":"2006","journal-title":"J. Commun. Manag."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"193","DOI":"10.5465\/amp.2013.0091","article-title":"Opening the black box of \u201cnot invented here\u201d: Attitudes, decision biases, and behavioral consequences","volume":"29","author":"Antons","year":"2015","journal-title":"Acad. Manag. Perspect."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Samset, K. (2010). Early Project Appraisal: Making the Initial Choices, Palgrave Macmillan.","DOI":"10.1057\/9780230289925"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13750-016-0079-2","article-title":"A framework for stakeholder engagement during systematic reviews and maps in environmental management","volume":"6","author":"Haddaway","year":"2017","journal-title":"Environ. Evid."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Abdollahpouri, H. (2019, January 27\u201328). Popularity Bias in Ranking and Recommendation. Proceedings of the 2019 AAAI\/ACM Conference on AI, Ethics, and Society, Honolulu, HI, USA.","DOI":"10.1145\/3306618.3314309"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1080\/00031305.2018.1518265","article-title":"Expert Knowledge Elicitation: Subjective, but Scientific","volume":"73","year":"2019","journal-title":"Am. Stat."},{"key":"ref_41","unstructured":"Nelson, J.D., and McKenzie, C.R. (2009). Encyclopaedia of Medical Decision Making, Sage."},{"key":"ref_42","unstructured":"Parnas, D.L., and Weiss, D.M. (1985, January 28\u201330). Active Design Reviews: Principles and Practices. Proceedings of the 8th International Conference on Software Engineering, London, UK."},{"key":"ref_43","first-page":"1","article-title":"What in the world were we thinking? Managing stakeholder expectations and engagement through transparent and collaborative project estimation","volume":"2","author":"Zuber","year":"2013","journal-title":"Proj. Manag. World J."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Loh, G.H. (2005, January 20\u201322). Simulation differences between academia and industry: A branch prediction case study. Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software, ISPASS 2005, Madison, WI, USA.","DOI":"10.1109\/ISPASS.2005.1430556"},{"key":"ref_45","first-page":"65","article-title":"Cognitive biases and the emotional disorders","volume":"3","author":"Mineka","year":"1992","journal-title":"Sage J."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"638","DOI":"10.1108\/MD-07-2019-1006","article-title":"Cognitive biases and decision-making strategies in times of change: A systematic literature review","volume":"59","author":"Acciarini","year":"2021","journal-title":"Manag. Decis."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1111\/1467-6486.00157","article-title":"Cognitive biases and strategic decision processes: An integrative perspective","volume":"36","author":"Das","year":"1999","journal-title":"J. Manag. Stud."},{"key":"ref_48","first-page":"137","article-title":"Even swaps: A rational method for making trade-offs","volume":"76","author":"Hammond","year":"1998","journal-title":"Harv. Bus. Rev."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1016\/j.ejor.2019.09.004","article-title":"On preference elicitation processes which mitigate the accumulation of biases in multi-criteria decision analysis","volume":"282","author":"Lahtinen","year":"2020","journal-title":"Eur. J. Oper. Res."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Cooke, R.M. (1991). Experts in Uncertainty: Opinion and Subjective Probability in Science, Oxford University Press.","DOI":"10.1093\/oso\/9780195064650.001.0001"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"O\u2019Hagan, A., Buck, C.E., Daneshkhah, A., Eiser, J.R., Garthwaite, P.H., Jenkinson, D.J., and Rakow, T. (2006). Uncertain Judgements: Eliciting Experts\u2019 Probabilities, John Wiley & Sons.","DOI":"10.1002\/0470033312"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Van Bossuyt, D.L., Beery, P., O\u2019Halloran, B.M., Hernandez, A., and Paulo, E. (2019). The Naval Postgraduate School\u2019s Department of Systems Engineering approach to mission engineering education through capstone projects. Systems, 7.","DOI":"10.3390\/systems7030038"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"724","DOI":"10.1002\/sys.21559","article-title":"Operationalizing digital twins through model-based systems engineering methods","volume":"23","author":"Bickford","year":"2020","journal-title":"Syst. Eng."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Bhatia, G., and Mesmer, B. (2019). Trends in occurrences of systems engineering topics in literature. Systems, 7.","DOI":"10.3390\/systems7020028"}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/11\/10\/499\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:01:09Z","timestamp":1760130069000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/11\/10\/499"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,28]]},"references-count":54,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["systems11100499"],"URL":"https:\/\/doi.org\/10.3390\/systems11100499","relation":{},"ISSN":["2079-8954"],"issn-type":[{"value":"2079-8954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,28]]}}}