{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T07:57:13Z","timestamp":1772265433597,"version":"3.50.1"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"S1","license":[{"start":{"date-parts":[[2022,7,1]],"date-time":"2022-07-01T00:00:00Z","timestamp":1656633600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,7,7]],"date-time":"2022-07-07T00:00:00Z","timestamp":1657152000000},"content-version":"vor","delay-in-days":6,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100008460","name":"National Center for Complementary and Integrative Health","doi-asserted-by":"publisher","award":["R01AT009457"],"award-info":[{"award-number":["R01AT009457"]}],"id":[{"id":"10.13039\/100008460","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Dietary supplements (DS) have been widely used by consumers, but the information around the efficacy and safety of DS is disparate or incomplete, thus creating barriers for consumers to find information effectively. Conversational agent (CA) systems have been applied to healthcare domain, but there is no such system to answer consumers regarding DS use, although widespread use of DS. In this study, we develop the first CA system for DS use.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>Our CA system for DS use developed on the MindMeld framework, consists of three components: question understanding, DS knowledge base, and answer generation. We collected and annotated 1509 questions to develop a\u00a0natural language understanding module (e.g., question type classifier, named entity recognizer) which was then integrated into MindMeld framework. CA then queries the DS knowledge base (i.e., iDISK) and generates answers using rule-based slot filling techniques. We evaluated the\u00a0algorithms of each component and the CA system as a whole.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>CNN is the best question classifier with an\u00a0F1 score of 0.81, and CRF is the best named entity recognizer with an\u00a0F1 score of 0.87. The system achieves an overall accuracy of 81% and an average score of 1.82 with succ@3\u2009+\u2009score of 76.2% and succ@2\u2009+\u2009of 66% approximately.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p>This study develops the first CA system for DS use using the\u00a0MindMeld framework and iDISK domain knowledge base.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-022-01888-5","type":"journal-article","created":{"date-parts":[[2022,7,7]],"date-time":"2022-07-07T12:03:12Z","timestamp":1657195392000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["A conversational agent system for dietary supplements use"],"prefix":"10.1186","volume":"22","author":[{"given":"Esha","family":"Singh","sequence":"first","affiliation":[]},{"given":"Anu","family":"Bompelli","sequence":"additional","affiliation":[]},{"given":"Ruyuan","family":"Wan","sequence":"additional","affiliation":[]},{"given":"Jiang","family":"Bian","sequence":"additional","affiliation":[]},{"given":"Serguei","family":"Pakhomov","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8258-3585","authenticated-orcid":false,"given":"Rui","family":"Zhang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,7,7]]},"reference":[{"key":"1888_CR1","unstructured":"National Institute of Health Sciences, Office of Dietary Supplements: What You Need to Know, Dietary Supplements. https:\/\/ods.od.nih.gov\/factsheets\/WYNTK-Consumer (2020). Accessed 3 Sep 2020."},{"key":"1888_CR2","unstructured":"Natural Medicines. https:\/\/naturalmedicines.therapeuticresearch.com\/. Accessed 18 Sept 2020."},{"key":"1888_CR3","unstructured":"Memorial Sloan Kettering Cancer Center https:\/\/www.mskcc.org\/cancer-care\/diagnosis-treatment\/symptom-management\/integrative-medicine\/herbs\/search. Accessed September 18, 2020."},{"key":"1888_CR4","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocz216","author":"R Rizvi","year":"2020","unstructured":"Rizvi R, Vasilakes J, Adam T, Melton G, Bishop J, Bian J, Tao C, Zhang R. idisk: the integrated dietary supplements knowledge base. J Am Med Inf Assoc JAMIA. 2020. https:\/\/doi.org\/10.1093\/jamia\/ocz216.","journal-title":"J Am Med Inf Assoc JAMIA"},{"key":"1888_CR5","doi-asserted-by":"publisher","first-page":"1032","DOI":"10.1016\/j.jbi.2011.08.008","volume":"44","author":"F Liu","year":"2011","unstructured":"Liu F, Antieau L, Yu H. Toward automated consumer question answering: automatically separating consumer questions from professional questions in the healthcare domain. J Biomed Inf. 2011;44:1032\u20138. https:\/\/doi.org\/10.1016\/j.jbi.2011.08.008.","journal-title":"J Biomed Inf"},{"key":"1888_CR6","doi-asserted-by":"publisher","first-page":"024","DOI":"10.1093\/jamia\/ocw024","volume":"23","author":"K Roberts","year":"2016","unstructured":"Roberts K, Demner-Fushman D. Interactive use of online health resources: a comparison of consumer and professional questions. J Am Med Inf Assoc. 2016;23:024. https:\/\/doi.org\/10.1093\/jamia\/ocw024.","journal-title":"J Am Med Inf Assoc"},{"key":"1888_CR7","unstructured":"BiTeM group: EAGLi platform-Question Answering in MEDLINE. http:\/\/eagl.unige.ch\/EAGLi\/. Accessed 14 Sep 2020."},{"key":"1888_CR8","unstructured":"LODQA: Question-Answering Over Linked Open Data. http:\/\/www.lodqa.org\/docs\/intro\/. Accessed 14 Sept 2020."},{"key":"1888_CR9","unstructured":"PACT Care BV: Florence-Your health assistant (2019). https:\/\/florence.chat\/. Accessed 14 Sep 2020."},{"key":"1888_CR10","unstructured":"Clue. https:\/\/helloclue.com\/. Accessed 10 April 2020."},{"key":"1888_CR11","unstructured":"Nutrition advice automation platform. https:\/\/getforksy.com\/. Accessed 14 Sep 2020."},{"key":"1888_CR12","unstructured":"Woebot Health: Try Woebot. https:\/\/woebothealth.com\/try-woebot\/. Accessed 14 Sep 2020."},{"key":"1888_CR13","unstructured":"ABBI the Artificial Intelligence Medical Marijuana, Facebook. https:\/\/www.abbibot.com\/. Accessed 14 Sep 2020."},{"key":"1888_CR14","unstructured":"Conversational Agent Definition. https:\/\/deepai.org\/machine-learning-glossary-and-terms\/conversational-agent. Accessed 15 Sep 2020."},{"key":"1888_CR15","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-15035-8.93","author":"S Hussain","year":"2019","unstructured":"Hussain S, Sianaki O, Ababneh N. A survey on conversational agents chatbots classification and design techniques. Science. 2019. https:\/\/doi.org\/10.1007\/978-3-030-15035-8.93.","journal-title":"Science"},{"key":"1888_CR16","unstructured":"S, tefania, B. Chatbot categories and their limitations. https:\/\/dzone.com\/articles\/chatbots-categories-and-their-limitations-1. (2018). Accessed 18 Sep 2020."},{"key":"1888_CR17","doi-asserted-by":"publisher","DOI":"10.1145\/3166054.3166058","author":"H Chen","year":"2017","unstructured":"Chen H, Liu X, Yin D, Tang J. A survey on dialogue systems: Recent advances and new frontiers. ACM SIGKDD Explor Newslett. 2017. https:\/\/doi.org\/10.1145\/3166054.3166058.","journal-title":"ACM SIGKDD Explor Newslett"},{"key":"1888_CR18","doi-asserted-by":"crossref","unstructured":"Yan Z, Duan N, Chen P, Zhou M, Zhou J, Li Z. Building task-oriented dialogue systems for online shopping. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. AAAI\u201917. AAAI Press; 2017. p. 4618\u20134625.","DOI":"10.1609\/aaai.v31i1.11182"},{"key":"1888_CR19","doi-asserted-by":"publisher","unstructured":"Spillane B, Gilmartin E, Saam C, Su K, Cowan BR, Lawless S, Wade V. Introducing adele: a personalized intelligent companion. In: Proceedings of the 1st ACM SIGCHI International Workshop on Investigating Social Interactions with Artificial Agents. ISIAA. Association for Computing Machinery, New York, NY, USA; 2017, p. 43\u201344. https:\/\/doi.org\/10.1145\/3139491.3139492. https:\/\/doi.org\/10.1145\/3139491.3139492","DOI":"10.1145\/3139491.3139492 10.1145\/3139491.3139492"},{"key":"1888_CR20","first-page":"137","volume":"5","author":"F Morbini","year":"2012","unstructured":"Morbini F, Forbell E, DeVault D, Sagae K, Traum D, Rizzo A. A mixed-initiative conversational dialogue system for healthcare. Science. 2012;5:137\u20139.","journal-title":"Science"},{"key":"1888_CR21","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1007\/978-3-319-47665-011","volume":"10011","author":"E Kimani","year":"2016","unstructured":"Kimani E, Bickmore T, Trinh H, Ring L, Paasche-Orlow M, Magnani J. A smartphone-based virtual agent for atrial fibrillation education and counseling. Science. 2016;10011:120\u20137. https:\/\/doi.org\/10.1007\/978-3-319-47665-011.","journal-title":"Science"},{"key":"1888_CR22","doi-asserted-by":"publisher","first-page":"547","DOI":"10.1016\/S2215-0366(18)30230-X","volume":"5","author":"D Sachan","year":"2018","unstructured":"Sachan D. Self-help robots drive blues away. Lancet Psychiatry. 2018;5:547. https:\/\/doi.org\/10.1016\/S2215-0366(18)30230-X.","journal-title":"Lancet Psychiatry"},{"key":"1888_CR23","doi-asserted-by":"publisher","first-page":"1125","DOI":"10.2196\/15360","volume":"21","author":"AB Kocaballi","year":"2019","unstructured":"Kocaballi AB, Berkovsky S, Quiroz J, Laranjo L, Tong HL, Rezazadegan D, Briatore A, Coiera E. The personalization of conversational agents in health care: systematic review. J Med Internet Res. 2019;21:1125.","journal-title":"J Med Internet Res"},{"key":"1888_CR24","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocz152","author":"D Demner-Fushman","year":"2019","unstructured":"Demner-Fushman D, Mrabet Y, Ben AA. Consumer health information and question answering: helping consumers find answers to their health-related information needs. J Am Med Inf Assoc JAMIA. 2019. https:\/\/doi.org\/10.1093\/jamia\/ocz152.","journal-title":"J Am Med Inf Assoc JAMIA"},{"key":"1888_CR25","unstructured":"Conversational AI platform for deep-domain voice interfaces and chatbots. 2021. https:\/\/www.mindmeld.com\/docs\/index.html."},{"key":"1888_CR26","unstructured":"Yahoo! Answers Homepage. https:\/\/answers.yahoo.com\/ Accessed 17 Sep 2020."},{"key":"1888_CR27","doi-asserted-by":"publisher","first-page":"276","DOI":"10.11613\/BM.2012.031","volume":"22","author":"M McHugh","year":"2012","unstructured":"McHugh M. Interrater reliability: the kappa statistic. Biochemia medica \u02c7casopis Hrvatskoga dru\u02c7stva medicinskih biokemi\u02c7cara HDMB. 2012;22:276\u201382. https:\/\/doi.org\/10.11613\/BM.2012.031.","journal-title":"Biochemia medica \u02c7casopis Hrvatskoga dru\u02c7stva medicinskih biokemi\u02c7cara HDMB"},{"key":"1888_CR28","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162","author":"J Pennington","year":"2014","unstructured":"Pennington J, Socher R, Manning C. Glove: global vectors for word representation. Science. 2014. https:\/\/doi.org\/10.3115\/v1\/D14-1162.","journal-title":"Science"},{"key":"1888_CR29","first-page":"5882","volume":"1408","author":"Y Kim","year":"2014","unstructured":"Kim Y. Convolutional neural networks for sentence classification. Science. 2014;1408:5882.","journal-title":"Science"},{"key":"1888_CR30","unstructured":"Spacy Homepage:. https:\/\/spacy.io\/. Accessed 14 Sep 2020."},{"key":"1888_CR31","unstructured":"Inside\u2013outside\u2013beginning (tagging). https:\/\/en.wikipedia.org\/wiki\/Inside%E2%80%93outside%E2%80%93beginning_(tagging). Accessed 13 Sep 2020."},{"key":"1888_CR32","first-page":"4805","volume":"1810","author":"J Devlin","year":"2019","unstructured":"Devlin J, Chang M-W, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. Science. 2019;1810:4805.","journal-title":"Science"},{"key":"1888_CR33","first-page":"01991","volume":"1508","author":"Z Huang","year":"2015","unstructured":"Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging. Science. 2015;1508:01991.","journal-title":"Science"},{"key":"1888_CR34","doi-asserted-by":"crossref","unstructured":"Zhou G, Jian S. Named entity recognition using an hmm-based chunk tagger, pp. 473\u2013480 (2002).","DOI":"10.3115\/1073083.1073163"},{"key":"1888_CR35","unstructured":"ElasticSearch. https:\/\/www.elastic.co\/elasticsearch\/service?ultron=EL-B-Stack-Trials-EMEA-UK-Exact&gambit=Elasticsearch-Core&blade=adwords-s&thor=elasticsearch&gclid=CjwKCAjw74b7BRA_EiwAF8yHFCqb_qahUf6zkAVxJknzYSqrxh7OFtdpIIBNAqOoO3CysanrtArTJxoCjEwQAvD_BwE. Accessed 18 Sep 2020."},{"key":"1888_CR36","unstructured":"Working with the Knowledge Base and Question Answerer. https:\/\/www.mindmeld.com\/docs\/userguide\/kb.html."},{"key":"1888_CR37","unstructured":"Overview of the TREC 2015 LiveQA Track. https:\/\/trec.nist.gov\/pubs\/trec24\/papers\/Overview-QA.pdf. Accessed 17 Sep 2020."},{"key":"1888_CR38","unstructured":"Mean reciprocal rank. https:\/\/en.wikipedia.org\/wiki\/Mean_reciprocal_rank. Accessed 18 April 2021."},{"key":"1888_CR39","first-page":"11054","volume":"2","author":"A Venkatesh","year":"2018","unstructured":"Venkatesh A, Khatri C, Ram A, Guo F, Gabriel R, Nagar A, Prasad R, Hedayatnia B, Metallinou A, Goel R, Yang S, Raju A. On evaluating and comparing conversational agents. Science. 2018;2:11054.","journal-title":"Science"},{"key":"1888_CR40","first-page":"00364","volume":"2008","author":"Q Li","year":"2020","unstructured":"Li Q, Peng H, Li J, Xia C, Yang R, Sun L, Yu PS, He L. A survey on text classification: from shallow to deep learning. Science. 2020;2008:00364.","journal-title":"Science"},{"key":"1888_CR41","doi-asserted-by":"publisher","first-page":"5841","DOI":"10.3390\/app10175841","volume":"10","author":"B Jang","year":"2020","unstructured":"Jang B, Kim M, Harerimana G, Kang S-U, Kim J. Bi-lstm model to increase accuracy in text classification: combining word2vec cnn and attention mechanism. Appl Sci. 2020;10:5841. https:\/\/doi.org\/10.3390\/app10175841.","journal-title":"Appl Sci"},{"key":"1888_CR42","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2019.01.078","author":"G Liu","year":"2019","unstructured":"Liu G, Guo J. Bidirectional lstm with attention mechanism and convolutional layer for text classification. Neurocomputing. 2019. https:\/\/doi.org\/10.1016\/j.neucom.2019.01.078.","journal-title":"Neurocomputing"},{"key":"1888_CR43","unstructured":"Berg H, Dalianis H. Augmenting a de-identification system for Swedish clinical text using open resources and deep learning. In: Proceedings of the Workshop on NLP and Pseudonymisation, Link\u00a8oping Electronic Press, Turku, Finland; 2019, p. 8\u201315. https:\/\/www.aclweb.org\/anthology\/W19-6502."},{"key":"1888_CR44","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1186\/s40537-020-00351-4","volume":"7","author":"J Leevy","year":"2020","unstructured":"Leevy J, Khoshgoftaar T, Villanustre F. Survey on rnn and crf models for de-identification of medical free text. J Big Data. 2020;7:73. https:\/\/doi.org\/10.1186\/s40537-020-00351-4.","journal-title":"J Big Data"},{"key":"1888_CR45","first-page":"03475","volume":"1606","author":"F Dernoncourt","year":"2016","unstructured":"Dernoncourt F, Lee JY, Uzuner O, Szolovits P. De-identification of patient notes with recurrent. Neural Netw. 2016;1606:03475.","journal-title":"Neural Netw"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-022-01888-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-022-01888-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-022-01888-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,7]],"date-time":"2022-07-07T12:05:22Z","timestamp":1657195522000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-022-01888-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7]]},"references-count":45,"journal-issue":{"issue":"S1","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["1888"],"URL":"https:\/\/doi.org\/10.1186\/s12911-022-01888-5","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7]]},"assertion":[{"value":"27 April 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 May 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 July 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"No ethics approval was required for the study.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"153"}}