{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T22:22:54Z","timestamp":1782858174179,"version":"3.54.5"},"reference-count":61,"publisher":"SAGE Publications","issue":"6","license":[{"start":{"date-parts":[[2023,4,28]],"date-time":"2023-04-28T00:00:00Z","timestamp":1682640000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"NASA Human Research Program","award":["80NSSC19K0654"],"award-info":[{"award-number":["80NSSC19K0654"]}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Hum Factors"],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>The objective of this study was to estimate trust from conversations using both lexical and acoustic data.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>As NASA moves to long-duration space exploration operations, the increasing need for cooperation between humans and virtual agents requires real-time trust estimation by virtual agents. Measuring trust through conversation is a novel and unintrusive approach.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Method<\/jats:title>\n                    <jats:p>A 2 (reliability) \u00d7 2 (cycles) \u00d7 3 (events) within-subject study with habitat system maintenance was designed to elicit various levels of trust in a conversational agent. Participants had trust-related conversations with the conversational agent at the end of each decision-making task. To estimate trust, subjective trust ratings were predicted using machine learning models trained on three types of conversational features (i.e., lexical, acoustic, and combined). After training, model explanation was performed using variable importance and partial dependence plots.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Results showed that a random forest algorithm, trained using the combined lexical and acoustic features, predicted trust in the conversational agent most accurately [Formula: see text]. The most important predictors were a combination of lexical and acoustic cues: average sentiment considering valence shifters, the mean of formants, and Mel-frequency cepstral coefficients (MFCC). These conversational features were identified as partial mediators predicting people\u2019s trust.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Precise trust estimation from conversation requires lexical cues and acoustic cues.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Application<\/jats:title>\n                    <jats:p>These results showed the possibility of using conversational data to measure trust, and potentially other dynamic mental states, unobtrusively and dynamically.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/00187208231166624","type":"journal-article","created":{"date-parts":[[2023,4,28]],"date-time":"2023-04-28T14:19:12Z","timestamp":1682691552000},"page":"1724-1741","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":18,"title":["It\u2019s Not Only What You Say, But Also How You Say It: Machine Learning Approach to Estimate Trust from Conversation"],"prefix":"10.1177","volume":"66","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0819-4693","authenticated-orcid":false,"given":"Mengyao","family":"Li","sequence":"first","affiliation":[{"name":"Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, Wisconsin, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Isabel M","family":"Erickson","sequence":"additional","affiliation":[{"name":"Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, Wisconsin, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ernest V","family":"Cross","sequence":"additional","affiliation":[{"name":"TRACLabs, Webster, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John D","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, Wisconsin, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2023,4,28]]},"reference":[{"key":"bibr1-00187208231166624","volume-title":"Measuring trust: A text analysis approach to compare, contrast, and select trust questionnaires","author":"Alsaid A.","year":"2022"},{"key":"bibr2-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-020-00694-1"},{"key":"bibr3-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.51.6.1173"},{"key":"bibr4-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0185651"},{"key":"bibr5-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1177\/0018720813482327"},{"key":"bibr6-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1063\/5.0066503"},{"key":"bibr7-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1121\/1.4714351"},{"key":"bibr8-00187208231166624","volume-title":"Wrassp: An R wrapper to the ASSP Library","author":"Bombien L.","year":"2021"},{"key":"bibr9-00187208231166624","volume-title":"Research on negotiation in organizations","author":"Bromiley P.","year":"1995"},{"key":"bibr10-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0031353"},{"key":"bibr11-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1177\/0018720816649094"},{"issue":"0","key":"bibr12-00187208231166624","first-page":"1","volume":"00","author":"Chiou E. K.","year":"2021","journal-title":"Human Factors"},{"key":"bibr13-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1111\/cogs.12009"},{"key":"bibr14-00187208231166624","first-page":"73","author":"Desai M.","year":"2012","journal-title":"HRI\u201912\u2014Proceedings of the 7th Annual ACM\/IEEE International Conference on Human-Robot Interaction"},{"key":"bibr15-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1518\/001872006778606822"},{"key":"bibr16-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1007\/s10726-012-9339-x"},{"key":"bibr17-00187208231166624","volume-title":"Human-AI Teaming: State-of-the-Art and Research Needs (Issue December)","author":"Endsley M. R.","year":"2021"},{"key":"bibr18-00187208231166624","unstructured":"Gauder L., Pepino L., Riera P., Brussino S., Vidal J., Gravano A., Ferrer L. (2021). A Study on the manifestation of trust in speech. ArXiv Preprint, 1\u201331."},{"key":"bibr19-00187208231166624","first-page":"3","author":"Goudbeek M.","year":"2009","journal-title":"Interspeech2009"},{"key":"bibr20-00187208231166624","doi-asserted-by":"publisher","DOI":"10.32614\/RJ-2017-016"},{"key":"bibr21-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1007\/s11747-020-00753-z"},{"key":"bibr22-00187208231166624","first-page":"969","volume-title":"Proceedings of the Space Technology and Applications International Forum","author":"Izygon M.","year":"2008"},{"key":"bibr23-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1207\/S15327566IJCE0401_04"},{"key":"bibr24-00187208231166624","doi-asserted-by":"publisher","DOI":"10.5898\/JHRI.3.1.Johnson"},{"key":"bibr25-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-24571-8_48"},{"key":"bibr26-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1080\/17470218.2015.1091484"},{"key":"bibr27-00187208231166624","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2021.604977"},{"key":"bibr28-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1177\/0893318999132007"},{"key":"bibr29-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1145\/3530874"},{"key":"bibr30-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2015.10.020"},{"key":"bibr31-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1080\/00909882.2010.536844"},{"key":"bibr32-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1518\/hfes.46.1.50_30392"},{"key":"bibr33-00187208231166624","unstructured":"Li M., Alsaid A., Noejovich S. I., Cross E. V., Lee J. D. (2020). Towards a conversational measure of trust. AAAI Fall Symposium FSS-20\/SSS-20, 1\u20136."},{"key":"bibr35-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1145\/3342197.3344525"},{"key":"bibr34-00187208231166624","doi-asserted-by":"crossref","unstructured":"Li M., Kamaraj A. V., Lee J. D. (2023). Modeling Trust Dimensions and Dynamics in Human-Agent Conversation: A Trajectory Epistemic Network Analysis Approach. International Journal of Human-Computer Interaction.","DOI":"10.1080\/10447318.2023.2201555"},{"key":"bibr36-00187208231166624","doi-asserted-by":"publisher","DOI":"10.2307\/258792"},{"issue":"0","key":"bibr37-00187208231166624","first-page":"1","volume":"00","author":"McDonald A. D.","year":"2020","journal-title":"Human Factors: The Journal of the Human Factors and Ergonomics Society"},{"key":"bibr38-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1177\/0018720819856454"},{"key":"bibr39-00187208231166624","doi-asserted-by":"crossref","unstructured":"Mohammad S. M. (2018). Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 English words. 56th Annual Meeting of the Association for Computational Linguistics, 174\u2013184.","DOI":"10.18653\/v1\/P18-1017"},{"key":"bibr40-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1016\/j.evolhumbehav.2016.10.010"},{"issue":"6","key":"bibr41-00187208231166624","first-page":"4515","volume":"5","author":"Nalini N. J.","year":"2013","journal-title":"International Journal of Engineering and Technology"},{"key":"bibr42-00187208231166624","first-page":"93","author":"Nielsen F. \u00c5.","year":"2011","journal-title":"Big Things Come in Small Packages"},{"key":"bibr43-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1016\/j.leaqua.2010.03.002"},{"key":"bibr44-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1007\/s11135-006-9018-6"},{"key":"bibr45-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1177\/0018720820960865"},{"key":"bibr46-00187208231166624","volume-title":"R Foundation for Statistical Computing","author":"R Development Core Team R.","year":"2011"},{"key":"bibr47-00187208231166624","volume-title":"Package \u201csentimentr\u201d","author":"Rinker T.","year":"2017"},{"key":"bibr48-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1109\/AERO.2014.6836403"},{"key":"bibr49-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1142\/9789812775320_0021"},{"key":"bibr50-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2022.3160132"},{"key":"bibr51-00187208231166624","doi-asserted-by":"publisher","DOI":"10.3390\/electronics10111259"},{"key":"bibr52-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1111\/j.1460-2466.2011.01619.x"},{"key":"bibr53-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2019.106215"},{"key":"bibr54-00187208231166624","first-page":"252","volume-title":"Cognition and Multi-agent Interaction: From Cognitive Modeling to Social Smulation","author":"Trafton J. G.","year":"2006"},{"key":"bibr55-00187208231166624","doi-asserted-by":"crossref","unstructured":"Trainor L. J., Austin C. M., Desjardins R. N. (2000). Is Infant-Directed Speech Prosody a Result of the Vocal Expression of Emotion? https:\/\/doi.org\/10.1111\/1467-9280.00240.","DOI":"10.1111\/1467-9280.00240"},{"key":"bibr56-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2008.11.007"},{"key":"bibr57-00187208231166624","doi-asserted-by":"publisher","DOI":"10.4337\/9781782547419.00037"},{"key":"bibr58-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1080\/14639220500370105"},{"issue":"0","key":"bibr59-00187208231166624","first-page":"1","volume":"00","author":"Yang X. J.","year":"2021","journal-title":"Human Factors: The Journal of the Human Factors and Ergonomics Society"},{"key":"bibr60-00187208231166624","doi-asserted-by":"publisher","DOI":"10.5334\/jors.160"},{"key":"bibr61-00187208231166624","doi-asserted-by":"publisher","DOI":"10.1080\/07350015.2019.1624293"}],"container-title":["Human Factors: The Journal of the Human Factors and Ergonomics Society"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/00187208231166624","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/00187208231166624","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/00187208231166624","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T07:53:54Z","timestamp":1777449234000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/00187208231166624"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,28]]},"references-count":61,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,6]]}},"alternative-id":["10.1177\/00187208231166624"],"URL":"https:\/\/doi.org\/10.1177\/00187208231166624","relation":{},"ISSN":["0018-7208","1547-8181"],"issn-type":[{"value":"0018-7208","type":"print"},{"value":"1547-8181","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,28]]}}}