{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T20:54:08Z","timestamp":1777668848046,"version":"3.51.4"},"reference-count":102,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2017,3,1]],"date-time":"2017-03-01T00:00:00Z","timestamp":1488326400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Hum Factors"],"published-print":{"date-parts":[[2017,3]]},"abstract":"<jats:sec>\n                    <jats:title>Objective:<\/jats:title>\n                    <jats:p>We have developed a framework for guiding measurement in human\u2013machine systems.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Background:<\/jats:title>\n                    <jats:p>The assessment of safety and performance in human\u2013machine systems often relies on direct measurement, such as tracking reaction time and accidents. However, safety and performance emerge from the combination of several variables. The assessment of precursors to safety and performance are thus an important part of predicting and improving outcomes in human\u2013machine systems.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Method:<\/jats:title>\n                    <jats:p>As part of an in-depth literature analysis involving peer-reviewed, empirical articles, we located and classified variables important to human\u2013machine systems, giving a snapshot of the state of science on human\u2013machine system safety and performance. Using this information, we created a framework of safety and performance in human\u2013machine systems.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results:<\/jats:title>\n                    <jats:p>This framework details several inputs and processes that collectively influence safety and performance. Inputs are divided according to human, machine, and environmental inputs. Processes are divided into attitudes, behaviors, and cognitive variables. Each class of inputs influences the processes and, subsequently, outcomes that emerge in human\u2013machine systems.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion:<\/jats:title>\n                    <jats:p>This framework offers a useful starting point for understanding the current state of the science and measuring many of the complex variables relating to safety and performance in human-machine systems.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Application:<\/jats:title>\n                    <jats:p>This framework can be applied to the design, development, and implementation of automated machines in spaceflight, military, and health care settings. We present a hypothetical example in our write-up of how it can be used to aid in project success.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/0018720817695077","type":"journal-article","created":{"date-parts":[[2017,3,21]],"date-time":"2017-03-21T19:28:50Z","timestamp":1490124530000},"page":"172-188","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":46,"title":["A Framework to Guide the Assessment of Human\u2013Machine Systems"],"prefix":"10.1177","volume":"59","author":[{"given":"Kimberly","family":"Stowers","sequence":"first","affiliation":[]},{"given":"James","family":"Oglesby","sequence":"additional","affiliation":[{"name":"University of Central Florida, Orlando"}]},{"given":"Shirley","family":"Sonesh","sequence":"additional","affiliation":[{"name":"Tulane University, New Orleans, Louisiana"}]},{"given":"Kevin","family":"Leyva","sequence":"additional","affiliation":[{"name":"Riptide Software, Orlando, Florida"}]},{"given":"Chelsea","family":"Iwig","sequence":"additional","affiliation":[]},{"given":"Eduardo","family":"Salas","sequence":"additional","affiliation":[{"name":"Rice University, Houston, TX"}]}],"member":"179","published-online":{"date-parts":[[2017,3,21]]},"reference":[{"key":"bibr1-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.ssci.2005.04.006"},{"key":"bibr2-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1007\/s11229-014-0645-5"},{"key":"bibr3-0018720817695077","volume-title":"Understanding attitudes and predicting social behaviour","author":"Ajzen I.","year":"1980"},{"key":"bibr4-0018720817695077","unstructured":"American Psychological Association. (2016). Personality. Retrieved from http:\/\/www.apa.org\/topics\/personality\/"},{"key":"bibr5-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1207\/s15327043hup1101_3"},{"key":"bibr6-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2008.06.001"},{"key":"bibr7-0018720817695077","volume-title":"Aviation automation: The search for a human-centered approach","author":"Billings C. E.","year":"1997"},{"key":"bibr8-0018720817695077","unstructured":"Billman D., Feary M., Rochlis-Zumbado J. (2011). Evidence report: Risk of inadequate design of human and automation\/robotic integration. Retrieved from https:\/\/ntrs.nasa.gov\/archive\/nasa\/casi.ntrs.nasa.gov\/20110014661.pdf"},{"key":"bibr9-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1023\/B:GRUP.0000021840.85686.57"},{"key":"bibr10-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.fusengdes.2013.02.064"},{"key":"bibr11-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1037\/a0018171"},{"key":"bibr12-0018720817695077","doi-asserted-by":"publisher","DOI":"10.2307\/258353"},{"key":"bibr13-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1177\/0018720811435843"},{"key":"bibr14-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1080\/08995600802115904"},{"key":"bibr15-0018720817695077","doi-asserted-by":"publisher","DOI":"10.21236\/ADA600351"},{"key":"bibr16-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2015.05.007"},{"key":"bibr17-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1080\/00140130600980888"},{"key":"bibr18-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2004.11.008"},{"key":"bibr19-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCA.2010.2084082"},{"key":"bibr20-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.apergo.2013.02.010"},{"key":"bibr21-0018720817695077","unstructured":"Daines G. (2015). NASA\u2019s journey to Mars. Retrieved from http:\/\/www.nasa.gov\/content\/nasas-journey-to-mars"},{"key":"bibr22-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1177\/1555343411410160"},{"key":"bibr23-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/S1071-5819(03)00039-9"},{"key":"bibr24-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1201\/b12474"},{"key":"bibr25-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2012.01.004"},{"key":"bibr26-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.aap.2006.10.003"},{"key":"bibr27-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/S1071-5819(03)00038-7"},{"key":"bibr28-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1177\/154193129904300345"},{"key":"bibr29-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1177\/106480460000800102"},{"key":"bibr30-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1109\/NAECON.1988.195097"},{"key":"bibr31-0018720817695077","first-page":"287","volume":"3","author":"Endsley M. R.","year":"1995","journal-title":"Human Factors in Aviation Operations"},{"key":"bibr32-0018720817695077","first-page":"163","volume-title":"Automation and human performance: Theory and applications","author":"Endsley M. R.","year":"1996"},{"key":"bibr33-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1201\/b12461"},{"key":"bibr34-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1201\/b11371"},{"key":"bibr35-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1080\/001401399185595"},{"key":"bibr36-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1518\/001872095779064555"},{"key":"bibr37-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCA.2005.855783"},{"key":"bibr38-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCA.2011.2181162"},{"key":"bibr39-0018720817695077","volume-title":"A meta-analysis of factors influencing the development of human\u2013robot trust","author":"Hancock P. A.","year":"2011"},{"key":"bibr40-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1518\/001872007X230226"},{"key":"bibr41-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1177\/001872088903100503"},{"key":"bibr42-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.intcom.2005.09.007"},{"key":"bibr43-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1177\/155534340700100102"},{"key":"bibr44-0018720817695077","volume-title":"Resilience engineering: Concepts and precepts","author":"Hollnagel E.","year":"2007"},{"key":"bibr45-0018720817695077","first-page":"259","volume-title":"The persuasion handbook","author":"Householder H. J.","year":"2003"},{"key":"bibr46-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1002\/prs.680160304"},{"key":"bibr47-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.ergon.2006.01.008"},{"key":"bibr48-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1518\/001872005775570989"},{"key":"bibr49-0018720817695077","unstructured":"Kirlik A. (1993). Acquisition and production of skilled behavior in dynamic decision-making tasks. Retrieved from https:\/\/ntrs.nasa.gov\/search.jsp?R=19920007914"},{"key":"bibr50-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.ergon.2009.04.002"},{"key":"bibr51-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1162\/105474602760204282"},{"key":"bibr52-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1518\/hfes.46.1.50_30392"},{"key":"bibr53-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2008.05.001"},{"key":"bibr54-0018720817695077","volume-title":"Engineering a safer world: Systems thinking applied to safety","author":"Leveson N.","year":"2011"},{"key":"bibr55-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.actaastro.2012.04.004"},{"key":"bibr56-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.ergon.2012.09.001"},{"key":"bibr57-0018720817695077","doi-asserted-by":"publisher","DOI":"10.2307\/259182"},{"key":"bibr58-0018720817695077","doi-asserted-by":"publisher","DOI":"10.2307\/258792"},{"key":"bibr59-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1207\/s15327043hup1004_1"},{"key":"bibr60-0018720817695077","unstructured":"McBride M., Morgan S. (2010). Trust calibration for automated decision aids. Retrieved from https:\/\/www.ihssnc.org\/portals\/0\/Documents\/VIMSDocuments\/McBride_Research_Brief.pdf"},{"key":"bibr61-0018720817695077","unstructured":"McCoppin R., Berger S. (2015, June 18). Poisoning death of Highland Park couple points to alarming trend. Chicago Tribune. Retrieved from http:\/\/www.chicagotribune.com\/suburbs\/lake-county-news-sun\/news\/ct-keyless-ignition-risk-met-20150618-story.html"},{"key":"bibr62-0018720817695077","unstructured":"Meister J. (2015). Survey shows amount of trust in autonomous vehicles. Retrieved from http:\/\/www.pddnet.com\/news\/2015\/10\/survey-shows-amount-trust-autonomous-vehicles"},{"key":"bibr63-0018720817695077","doi-asserted-by":"publisher","DOI":"10.21236\/ADA606748"},{"key":"bibr64-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1518\/001872007779598037"},{"key":"bibr65-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1177\/001872089606380211"},{"key":"bibr66-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1518\/001872008X250683"},{"key":"bibr67-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1007\/s10270-008-0084-1"},{"key":"bibr68-0018720817695077","unstructured":"National Aeronautics and Space Administration. (2011). Accident precursor analysis handbook. Retrieved from https:\/\/www.hq.nasa.gov\/office\/codeq\/doctree\/NASA_SP-2011-3423.pdf"},{"key":"bibr69-0018720817695077","unstructured":"National Aeronautics and Space Administration. (2016). NASA Technical Standards System (NTSS). Retrieved from https:\/\/standards.nasa.gov\/nasa-technical-standards"},{"key":"bibr70-0018720817695077","volume-title":"Usability engineering","author":"Nielsen J.","year":"1994"},{"key":"bibr71-0018720817695077","unstructured":"Oglesby J. M., Iwig C., Stowers K., Sonesh S., Leyva K., Salas E. (2017). Human\u2013automation systems safety and performance: A qualitative review of measures and metrics. Manuscript submitted for publication."},{"key":"bibr72-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1177\/1071181312561358"},{"key":"bibr73-0018720817695077","volume":"23","author":"Olsen D. R.","year":"2003","journal-title":"All Faculty Publications"},{"key":"bibr74-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1080\/08995600902768800"},{"key":"bibr75-0018720817695077","first-page":"45","volume-title":"Performance under stress","author":"Parasuraman R.","year":"2008"},{"key":"bibr76-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1177\/0018720810376055"},{"key":"bibr77-0018720817695077","first-page":"119","volume-title":"Automation technology and human performance: Current research and trends","author":"Parasuraman R.","year":"1999"},{"key":"bibr78-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1518\/001872096778827279"},{"key":"bibr79-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1518\/001872097778543886"},{"key":"bibr80-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1109\/3468.844354"},{"key":"bibr81-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1518\/155534308X284417"},{"key":"bibr82-0018720817695077","first-page":"13","volume-title":"Proceedings of the 3rd Annual Conference on Human\u2013Robot Interaction","author":"Pina P. E.","year":"2008"},{"key":"bibr83-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1207\/S15327108IJAP1004_6"},{"key":"bibr84-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2008.12.025"},{"key":"bibr85-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781139062367"},{"key":"bibr86-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1037\/h0031464"},{"key":"bibr87-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1007\/s13164-012-0129-9"},{"key":"bibr88-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1518\/001872095779049525"},{"key":"bibr89-0018720817695077","first-page":"239","volume-title":"Proceedings of the Eighth International Symposium on Aviation Psychology","author":"Sarter N. B.","year":"1995"},{"key":"bibr90-0018720817695077","first-page":"1926","volume-title":"Handbook of human factors and ergonomics","author":"Sarter N. B.","year":"1997"},{"key":"bibr91-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.apergo.2012.05.008"},{"key":"bibr92-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1518\/155723405783703082"},{"key":"bibr93-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1006\/ijhc.1999.0252"},{"key":"bibr94-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1145\/1121241.1121249"},{"key":"bibr95-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1177\/1541931213601392"},{"key":"bibr96-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1037\/a0024170"},{"key":"bibr97-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.37.3.395"},{"key":"bibr98-0018720817695077","unstructured":"Wood S. (2004). Flight crew reliance on automation (CAA Paper 10). Retrieved from https:\/\/publicapps.caa.co.uk\/docs\/33\/2004_10.PDF"},{"key":"bibr99-0018720817695077","volume-title":"Behind human error","author":"Woods D. D.","year":"2010"},{"key":"bibr100-0018720817695077","volume-title":"Behind human error: Cognitive systems, computers and hindsight","author":"Woods D. D.","year":"1994"},{"key":"bibr101-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1109\/PROC.1986.13634"},{"key":"bibr102-0018720817695077","doi-asserted-by":"publisher","DOI":"10.1016\/j.paid.2013.04.004"}],"container-title":["Human Factors: The Journal of the Human Factors and Ergonomics Society"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0018720817695077","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0018720817695077","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0018720817695077","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T07:51:20Z","timestamp":1777449080000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0018720817695077"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,3]]},"references-count":102,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2017,3]]}},"alternative-id":["10.1177\/0018720817695077"],"URL":"https:\/\/doi.org\/10.1177\/0018720817695077","relation":{},"ISSN":["0018-7208","1547-8181"],"issn-type":[{"value":"0018-7208","type":"print"},{"value":"1547-8181","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,3]]}}}