{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,31]],"date-time":"2026-01-31T00:59:11Z","timestamp":1769821151542,"version":"3.49.0"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"S4","license":[{"start":{"date-parts":[[2020,12,1]],"date-time":"2020-12-01T00:00:00Z","timestamp":1606780800000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2020,12,14]],"date-time":"2020-12-14T00:00:00Z","timestamp":1607904000000},"content-version":"vor","delay-in-days":13,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Previously, we introduced our Patient Health Information Dialogue Ontology (PHIDO) that manages the dialogue and contextual information of the session between an agent and a health consumer. In this study, we take the next step and introduce the Conversational Ontology Operator (COO), the software engine harnessing PHIDO. We also developed a question-answering subsystem called Frankenstein Ontology Question-Answering for User-centric Systems (FOQUS) to support the dialogue interaction.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>We tested both the dialogue engine and the question-answering system using application-based competency questions and questions furnished from our previous Wizard of OZ simulation trials.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Our results revealed that the dialogue engine is able to perform the core tasks of communicating health information and conversational flow. Inter-rater agreement and accuracy scores among four reviewers indicated perceived, acceptable responses to the questions asked by participants from the simulation studies, yet the composition of the responses was deemed mediocre by our evaluators.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>Overall, we present some preliminary evidence of a functioning ontology-based system to manage dialogue and consumer questions. Future plans for this work will involve deploying this system in a speech-enabled agent to assess its usage with potential health consumer users.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-020-01267-y","type":"journal-article","created":{"date-parts":[[2020,12,14]],"date-time":"2020-12-14T08:02:54Z","timestamp":1607932974000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Conversational ontology operator: patient-centric vaccine dialogue management engine for spoken conversational agents"],"prefix":"10.1186","volume":"20","author":[{"given":"Muhammad","family":"Amith","sequence":"first","affiliation":[]},{"given":"Rebecca Z.","family":"Lin","sequence":"additional","affiliation":[]},{"given":"Licong","family":"Cui","sequence":"additional","affiliation":[]},{"given":"Dennis","family":"Wang","sequence":"additional","affiliation":[]},{"given":"Anna","family":"Zhu","sequence":"additional","affiliation":[]},{"given":"Grace","family":"Xiong","sequence":"additional","affiliation":[]},{"given":"Hua","family":"Xu","sequence":"additional","affiliation":[]},{"given":"Kirk","family":"Roberts","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4267-1924","authenticated-orcid":false,"given":"Cui","family":"Tao","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,12,14]]},"reference":[{"issue":"2","key":"1267_CR1","first-page":"145","volume":"16","author":"D Litman","year":"2006","unstructured":"Litman D, Ros\u00e9 C, Forbes-Riley K, VanLehn K, Bhembe D, Silliman S. Spoken versus typed human and computer dialogue tutoring. Int J Artif Intell Educ. 2006; 16(2):145\u201370.","journal-title":"Int J Artif Intell Educ"},{"issue":"2","key":"1267_CR2","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1023\/A:1022334530417","volume":"6","author":"L Damianos","year":"2003","unstructured":"Damianos L, Loehr D, Burke C, Hansen S, Viszmeg M. The msiia experiment: Using speech to enhance human performance on a cognitive task. Int J Speech Technol. 2003; 6(2):133\u201344.","journal-title":"Int J Speech Technol"},{"issue":"4","key":"1267_CR3","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1023\/A:1020969227003","volume":"5","author":"S Harris","year":"2002","unstructured":"Harris S, Biermann A. Mouse selection versus voice selection of menu items. Int J Speech Technol. 2002; 5(4):389\u2013402.","journal-title":"Int J Speech Technol"},{"issue":"9","key":"1267_CR4","doi-asserted-by":"publisher","first-page":"1187","DOI":"10.1016\/j.vaccine.2016.01.023","volume":"34","author":"M Gilkey","year":"2016","unstructured":"Gilkey M, Calo W, Moss J, Shah P, Marciniak M, Brewer N. Provider communication and HPV vaccination: The impact of recommendation quality. Vaccine. 2016; 34(9):1187\u201392. https:\/\/doi.org\/10.1016\/j.vaccine.2016.01.023.","journal-title":"Vaccine"},{"key":"1267_CR5","unstructured":"Rimer B, Harper H, Witte O. Accelerating hpv vaccine uptake: urgency action to prevent cancer; a report to the president of the united states from the president\u2019s cancer panel. National Cancer Institute. 2014. https:\/\/deainfo.nci.nih.gov\/Advisory\/pcp\/annualReports\/HPV\/PDF\/PCP_Annual_Report_2012-2013.pdf. Accessed 23 July 2020."},{"key":"1267_CR6","unstructured":"U.S Department of Health and Human Services. Healthy People 2020 Immunization and Infectious Disease: 2020. https:\/\/www.healthypeople.gov\/2020\/topics-objectives\/topic\/immunization-and-infectious-diseases\/objectives. Accessed 23 July 2020."},{"issue":"1","key":"1267_CR7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.2200\/S00204ED1V01Y200910HLT005","volume":"2","author":"K Jokinen","year":"2009","unstructured":"Jokinen K, McTear M. Spoken dialogue systems. Synth Lect Hum Lang Technol. 2009; 2(1):1\u2013151.","journal-title":"Synth Lect Hum Lang Technol"},{"key":"1267_CR8","doi-asserted-by":"publisher","first-page":"17","DOI":"10.3233\/978-1-61499-951-5-17","volume":"257","author":"M Amith","year":"2019","unstructured":"Amith M, Zhu A, Cunningham R, Lin R, Savas L, Shay L, Chen Y, Gong Y, Boom J, Roberts K, Tao C. Early usability assessment of a conversational agent for hpv vaccination. Stud Health Technol Inform. 2019; 257:17\u201323. https:\/\/doi.org\/10.3233\/978-1-61499-951-5-17.","journal-title":"Stud Health Technol Inform"},{"key":"1267_CR9","first-page":"43","volume":"2020","author":"M Amith","year":"2020","unstructured":"Amith M, Lin R, Cunningham R, Wu Q, Savas L, Gong Y, Boom J, Tang L, Tao C. Examining potential usability and health beliefs among young adults using a conversational agent for hpv vaccine counseling. AMIA Jt Summits on Transl Sci Proc. AMIA Jt Summits Transl Sci. 2020; 2020:43\u201352.","journal-title":"AMIA Jt Summits on Transl Sci Proc. AMIA Jt Summits Transl Sci"},{"issue":"1","key":"1267_CR10","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1016\/0885-2308(91)90019-M","volume":"5","author":"N Fraser","year":"1991","unstructured":"Fraser N, Gilbert G. Simulating speech systems. Comput Speech Lang. 1991; 5(1):81\u201399.","journal-title":"Comput Speech Lang"},{"key":"1267_CR11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1017\/S0269888919000237","volume":"34","author":"A Olivares-Alarcos","year":"2019","unstructured":"Olivares-Alarcos A, Be\u00dfler D, Khamis A, Goncalves P, Habib MK, Bermejo-Alonso J, Barreto M, Diab M, Rosell J, Quintas J, et al. A review and comparison of ontology-based approaches to robot autonomy. Knowl Eng Rev. 2019; 34:1\u201329.","journal-title":"Knowl Eng Rev"},{"key":"1267_CR12","unstructured":"Giarratano J, Riley G. Expert Systems: Principles and Programming. 4th ed: Thomson Course Technology; 2004."},{"key":"1267_CR13","unstructured":"Abrams M. Lenny Kravitz at the Festhalle in Frankfurt Germany, March 20, 1996. \u24b8Stars and Stripes\/Micheal Abrams. https:\/\/www.stripes.com\/news\/for-lenny-kravitz-life-is-music-1.16294. Accessed 23 July 2020."},{"key":"1267_CR14","first-page":"432","volume":"225","author":"D Wang","year":"2016","unstructured":"Wang D, Cunningham R, Boom J, Amith M, Tao C. Towards a HPV vaccine knowledgebase for patient education content. Stud Health Technol Inform. 2016; 225:432\u20136.","journal-title":"Stud Health Technol Inform"},{"issue":"1","key":"1267_CR15","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1186\/s13326-015-0016-2","volume":"6","author":"M Amith","year":"2015","unstructured":"Amith M, Gong Y, Cunningham R, Boom J, Tao C. Developing viso: vaccine information statement ontology for patient education. J Biomed Semant. 2015; 6(1):23.","journal-title":"J Biomed Semant"},{"issue":"21","key":"1267_CR16","first-page":"1","volume":"20","author":"M Amith","year":"2019","unstructured":"Amith M, Roberts K, Tao C. Conceiving an application ontology to model patient human papillomavirus vaccine counseling for dialogue management. BMC Bioinformatics. 2019; 20(21):1\u201316.","journal-title":"BMC Bioinformatics"},{"issue":"3","key":"1267_CR17","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1007\/s10817-014-9305-1","volume":"53","author":"B Glimm","year":"2014","unstructured":"Glimm B, Horrocks I, Motik B, Stoilos G, Wang Z. Hermit: an owl 2 reasoner. J Autom Reason. 2014; 53(3):245\u201369.","journal-title":"J Autom Reason"},{"key":"1267_CR18","volume-title":"Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations","author":"C Manning","year":"2014","unstructured":"Manning C, Surdeanu M, Bauer J, Finkel J, Bethard S, McClosky D. The stanford corenlp natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Baltimore: Association for Computational Linguistics: 2014. p. 55\u201360."},{"key":"1267_CR19","volume-title":"4th European Semantic Web Conference ESWC","author":"E Kaufmann","year":"2007","unstructured":"Kaufmann E, Bernstein A, Fischer L. Nlp-reduce: A naive but domainindependent natural language interface for querying ontologies. In: 4th European Semantic Web Conference ESWC. Berlin: Springer: 2007. p. 1\u20132."},{"key":"1267_CR20","unstructured":"Morpha Stemmer. https:\/\/github.com\/knowitall\/morpha. Accessed 20 Apr 2019."},{"key":"1267_CR21","volume-title":"Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)","author":"R Speer","year":"2017","unstructured":"Speer R, Lowry-Duda J. ConceptNet at SemEval-2017 Task 2: Extending Word Embeddings with Multilingual Relational Knowledge. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017). Vancouver: Association for Computational Linguistics: 2017. p. 85\u20139."},{"key":"1267_CR22","volume-title":"Kdd, vol. 2","author":"A Monge","year":"1996","unstructured":"Monge A, Elkan C, et al. The field matching problem: Algorithms and applications. In: Kdd, vol. 2. Menlo Park: The AAAI Press: 1996. p. 267\u201370."},{"key":"1267_CR23","volume-title":"Proceedings of the SIGMOD 1997 Workshop on research issues on data mining and knowledge discovery","author":"A Monge","year":"2017","unstructured":"Monge A, Elkan C. An efficient domain-independent algorithm for detecting approximately duplicate database records. In: Proceedings of the SIGMOD 1997 Workshop on research issues on data mining and knowledge discovery. New York: Association for Computing Machinery: 2017. p. 23\u201329."},{"key":"1267_CR24","volume-title":"Extended Semantic Web Conference","author":"D Damljanovic","year":"2010","unstructured":"Damljanovic D, Agatonovic M, Cunningham H. Natural language interfaces to ontologies: Combining syntactic analysis and ontology-based lookup through the user interaction. In: Extended Semantic Web Conference. Berlin: Springer: 2010. p. 106\u201320."},{"key":"1267_CR25","unstructured":"Autayeu A. extJWNL. http:\/\/extjwnl.sourceforge.net\/. Accessed 20 June 2019."},{"key":"1267_CR26","unstructured":"Eclipse Foundation. Eclipse RDF4J. https:\/\/rdf4j.eclipse.org\/. Accessed 25 June 2019."},{"issue":"1","key":"1267_CR27","doi-asserted-by":"publisher","first-page":"11","DOI":"10.3233\/SW-2011-0025","volume":"2","author":"M Horridge","year":"2011","unstructured":"Horridge M, Bechhofer S. The owl api: A java api for owl ontologies. Semant Web. 2011; 2(1):11\u201321.","journal-title":"Semant Web"},{"key":"1267_CR28","unstructured":"Korstanje M. SimMetrics. https:\/\/github.com\/Simmetrics\/simmetrics. Accessed 25 June 2019."},{"key":"1267_CR29","unstructured":"Eclipse Foundation. Eclipse integrated development environment. https:\/\/eclipse.org\/. Accessed 25 June 2019."},{"issue":"1","key":"1267_CR30","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1177\/001316446002000104","volume":"20","author":"J Cohen","year":"1960","unstructured":"Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960; 20(1):37\u201346.","journal-title":"Educ Psychol Meas"},{"key":"1267_CR31","unstructured":"Advisory committee on immunization practices (ACIP). Meeting recommendations. 2019. https:\/\/www.cdc.gov\/vaccines\/acip\/index.html. Accessed 15 Aug 2019."},{"issue":"3","key":"1267_CR32","doi-asserted-by":"publisher","first-page":"342","DOI":"10.3758\/BF03195462","volume":"34","author":"R Corkrey","year":"2002","unstructured":"Corkrey R, Parkinson L. Interactive voice response: review of studies 1989\u20132000. Behav Res Methods Instrum Comput. 2002; 34(3):342\u201353.","journal-title":"Behav Res Methods Instrum Comput"},{"issue":"5","key":"1267_CR33","doi-asserted-by":"publisher","first-page":"556","DOI":"10.1016\/j.jbi.2005.12.004","volume":"39","author":"T Bickmore","year":"2006","unstructured":"Bickmore T, Giorgino T. Health dialog systems for patients and consumers. J Biomed Inform. 2006; 39(5):556\u201371.","journal-title":"J Biomed Inform"},{"issue":"2","key":"1267_CR34","doi-asserted-by":"publisher","first-page":"16","DOI":"10.2196\/jmir.1138","volume":"11","author":"R Neff","year":"2009","unstructured":"Neff R, Fry J. Periodic prompts and reminders in health promotion and health behavior interventions: systematic review. J Med Internet Res. 2009; 11(2):16.","journal-title":"J Med Internet Res"},{"issue":"1","key":"1267_CR35","doi-asserted-by":"publisher","first-page":"4","DOI":"10.2196\/jmir.1376","volume":"12","author":"T Webb","year":"2010","unstructured":"Webb T, Joseph J, Yardley L, Michie S. Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res. 2010; 12(1):4.","journal-title":"J Med Internet Res"},{"issue":"3","key":"1267_CR36","doi-asserted-by":"publisher","first-page":"80","DOI":"10.2196\/jmir.1893","volume":"14","author":"C Kennedy","year":"2012","unstructured":"Kennedy C, Powell J, Payne T, Ainsworth J, Boyd A, Buchan I. Active assistance technology for health-related behavior change: an interdisciplinary review. J Med Internet Res. 2012; 14(3):80.","journal-title":"J Med Internet Res"},{"key":"1267_CR37","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1093\/med:psych\/9780195165791.003.0007","volume":"2","author":"J Prochaska","year":"2005","unstructured":"Prochaska J, DiClemente C. The transtheoretical approach. Handb Psychother Integr. 2005; 2:147\u201371.","journal-title":"Handb Psychother Integr"},{"key":"1267_CR38","volume-title":"Motivational interviewing: preparing people to change addictive behavior","author":"W Miller","year":"1991","unstructured":"Miller W, Rollnick S. Motivational interviewing: preparing people to change addictive behavior. New York: Guilford Press; 1991."},{"key":"1267_CR39","doi-asserted-by":"publisher","first-page":"324","DOI":"10.1177\/109019817400200401","volume":"2","author":"M Becker","year":"1974","unstructured":"Becker M. The health belief model and personal health behavior. Health Educ Monogr. 1974; 2:324\u2013473.","journal-title":"Health Educ Monogr"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-020-01267-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s12911-020-01267-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-020-01267-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,12,14]],"date-time":"2020-12-14T08:10:38Z","timestamp":1607933438000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-020-01267-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12]]},"references-count":39,"journal-issue":{"issue":"S4","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["1267"],"URL":"https:\/\/doi.org\/10.1186\/s12911-020-01267-y","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12]]},"assertion":[{"value":"14 December 2020","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The data analyzed was derived from a previous experiment [] that was approved by The University of Texas Health Science Center\u2019s Committee for the Protection of Human Subjects (HSC-SBMI-17-0533).","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"Dr. Xu and The University of Texas Health Science Center at Houston have research-related financial interests in Melax Technologies, Inc.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"259"}}