{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T16:55:00Z","timestamp":1787504100664,"version":"build-2736575974"},"reference-count":55,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,10,9]],"date-time":"2020-10-09T00:00:00Z","timestamp":1602201600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,10,9]],"date-time":"2020-10-09T00:00:00Z","timestamp":1602201600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001665","name":"Agence Nationale de la Recherche","doi-asserted-by":"publisher","award":["ANR-16-CE23-0011"],"award-info":[{"award-number":["ANR-16-CE23-0011"]}],"id":[{"id":"10.13039\/501100001665","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Medical terminologies are commonly used in medicine. For instance, to answer a pharmacovigilance question, pharmacovigilance specialists (PVS) search in a pharmacovigilance database for reports in relation to a given drug. To do that, they first need to identify all MedDRA terms that might have been used to code an adverse reaction in the database, but terms may be numerous and difficult to select as they may belong to different parts of the hierarchy. In previous studies, three tools have been developed to help PVS identify and group all relevant MedDRA terms using three different approaches: forms, structured query-builder, and icons. Yet, a poor usability of the tools may increase PVS\u2019 workload and reduce their performance. This study aims to evaluate, compare and improve the three tools during two rounds of formative usability evaluation.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>First, a cognitive walkthrough was performed. Based on the design recommendations obtained from this evaluation, designers made modifications to their tools to improve usability. Once this re-engineering phase completed, six PVS took part in a usability test: difficulties, errors and verbalizations during their interaction with the three tools were collected. Their satisfaction was measured through the System Usability Scale. The design recommendations issued from the tests were used to adapt the tools.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>All tools had usability problems related to the lack of guidance in the graphical user interface (e.g., unintuitive labels). In two tools, the use of the SNOMED CT to find MedDRA terms hampered their use because French PVS were not used to it. For the most obvious and common terms, the icons-based interface would appear to be more useful. For the less frequently used MedDRA terms or those distributed in different parts of the hierarchy, the structured query-builder would be preferable thanks to its great power and flexibility. The form-based tool seems to be a compromise.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>These evaluations made it possible to identify the strengths of each tool but also their weaknesses to address them before further evaluation. Next step is to assess the acceptability of tools and the expressiveness of their results to help identify and group MedDRA terms.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12911-020-01280-1","type":"journal-article","created":{"date-parts":[[2020,10,9]],"date-time":"2020-10-09T09:03:00Z","timestamp":1602234180000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["How to interact with medical terminologies? Formative usability evaluations comparing three approaches for supporting the use of MedDRA by pharmacovigilance specialists"],"prefix":"10.1186","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7077-7267","authenticated-orcid":false,"given":"Romaric","family":"Marcilly","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Laura","family":"Douze","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6302-2333","authenticated-orcid":false,"given":"S\u00e9bastien","family":"Ferr\u00e9","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8550-8724","authenticated-orcid":false,"given":"Bissan","family":"Audeh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4239-8785","authenticated-orcid":false,"given":"Carlos","family":"Bobed","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8135-7340","authenticated-orcid":false,"given":"Agn\u00e8s","family":"Lillo-Le Lou\u00ebt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5477-180X","authenticated-orcid":false,"given":"Jean-Baptiste","family":"Lamy","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9775-2476","authenticated-orcid":false,"given":"C\u00e9dric","family":"Bousquet","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,10,9]]},"reference":[{"key":"1280_CR1","doi-asserted-by":"publisher","first-page":"518","DOI":"10.1055\/s-0038-1634545","volume":"37","author":"P Degoulet","year":"1998","unstructured":"Degoulet P, Sauquet D, Jaulent MC, Zapletal E, Lavril M. Rationale and design considerations for a semantic mediator in health information systems. Methods Inf Med. 1998;37:518\u201326.","journal-title":"Methods Inf Med"},{"key":"1280_CR2","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1197\/jamia.M1957","volume":"13","author":"ST Rosenbloom","year":"2006","unstructured":"Rosenbloom ST, Miller RA, Johnson KB, Elkin PL, Brown SH. Interface terminologies: facilitating direct entry of clinical data into electronic health record systems. J Am Med Inform Assoc. 2006;13:277\u201388.","journal-title":"J Am Med Inform Assoc"},{"key":"1280_CR3","doi-asserted-by":"publisher","first-page":"351","DOI":"10.1016\/j.ijmedinf.2011.09.010","volume":"81","author":"F Bakhshi-Raiez","year":"2012","unstructured":"Bakhshi-Raiez F, de Keizer NF, Cornet R, Dorrepaal M, Dongelmans D, Jaspers MWM. A usability evaluation of a SNOMED CT based compositional interface terminology for intensive care. Int J Med Inform. 2012;81:351\u201362.","journal-title":"Int J Med Inform"},{"key":"1280_CR4","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1016\/j.ijmedinf.2012.05.018","volume":"82","author":"MF Walji","year":"2013","unstructured":"Walji MF, Kalenderian E, Tran D, Kookal KK, Nguyen V, Tokede O, et al. Detection and characterization of usability problems in structured data entry interfaces in dentistry. Int J Med Inform. 2013;82:128\u201338.","journal-title":"Int J Med Inform"},{"key":"1280_CR5","unstructured":"World Health Organization. Essential medicines and health products - Pharmacovigilance. World Health Organization. Available from: https:\/\/www.who.int\/medicines\/areas\/quality_safety\/safety_efficacy\/pharmvigi\/en\/. [cited 2020 Jun 23]."},{"key":"1280_CR6","unstructured":"International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human (ICH). MedDRA - Medical Dictionnary for Regulatory Activities [Internet]. ICH harmonisation for better health. Available from: https:\/\/www.ich.org\/page\/meddra. [cited 2020 Jun 23]."},{"key":"1280_CR7","doi-asserted-by":"publisher","first-page":"109","DOI":"10.2165\/00002018-199920020-00002","volume":"20","author":"EG Brown","year":"1999","unstructured":"Brown EG, Wood SL. The Medical Dictionary for Regulatory Activities (MedDRA). Drug Saf. 1999;20:109\u201317.","journal-title":"Drug Saf"},{"key":"1280_CR8","doi-asserted-by":"publisher","first-page":"145","DOI":"10.2165\/00002018-200326030-00002","volume":"26","author":"EG Brown","year":"2003","unstructured":"Brown EG. Methods and pitfalls in searching drug safety databases Utilising the medical dictionary for regulatory activities (MedDRA). Drug Saf. 2003;26:145\u201358.","journal-title":"Drug Saf"},{"key":"1280_CR9","volume-title":"MedDRA\u00ae data retrieval and presentation: points to consider ICH \u2013 Endorsed Guide for MedDRA Users on Data Output. Release 3.19 Based on MedDRA Version 23.0","author":"MedDRA","year":"2020","unstructured":"MedDRA. MedDRA\u00ae data retrieval and presentation: points to consider ICH \u2013 Endorsed Guide for MedDRA Users on Data Output. Release 3.19 Based on MedDRA Version 23.0. 2020. Available from: https:\/\/admin.new.meddra.org\/sites\/default\/files\/guidance\/file\/000402_datretptc_r3_19_mar2020_10.pdf."},{"key":"1280_CR10","volume-title":"Introductory Guide for Standardised MedDRA Queries (SMQs) Version 23.1","author":"MedDRA","year":"2020","unstructured":"MedDRA. Introductory Guide for Standardised MedDRA Queries (SMQs) Version 23.1. 2020. Available from: https:\/\/admin.new.meddra.org\/sites\/default\/files\/guidance\/file\/SMQ_intguide_23_1_English.pdf."},{"key":"1280_CR11","doi-asserted-by":"publisher","first-page":"50","DOI":"10.3389\/fphar.2019.00050","volume":"10","author":"J Souvignet","year":"2019","unstructured":"Souvignet J, Declerck G, Trombert-Paviot B, Asfari H, Jaulent M-C, Bousquet C. Semantic queries expedite MedDRA terms selection thanks to a dedicated user Interface: a pilot study on five medical conditions. Front Pharmacol. 2019;10:50.","journal-title":"Front Pharmacol"},{"key":"1280_CR12","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1016\/j.jbi.2016.06.010","volume":"63","author":"J Souvignet","year":"2016","unstructured":"Souvignet J, Declerck G, Asfari H, Jaulent M-C, Bousquet C. OntoADR a semantic resource describing adverse drug reactions to support searching, coding, and information retrieval. J Biomed Inform. 2016;63:100\u20137.","journal-title":"J Biomed Inform"},{"key":"1280_CR13","unstructured":"International Organization for Standardization. ISO 9241-210:2010. Ergonomics of human-system interaction -- Part 210: Human-centred design for interactive systems. Geneva: International Organization for Standardization; 2010."},{"key":"1280_CR14","volume-title":"Reconciling Expressivity and Usability in Information Access, from File Systems to the Semantic Web - Habilitation \u00e0 Diriger les Recherches (HDR)","author":"S Ferr\u00e9","year":"2014","unstructured":"Ferr\u00e9 S. Reconciling Expressivity and Usability in Information Access, from File Systems to the Semantic Web - Habilitation \u00e0 Diriger les Recherches (HDR). Rennes: Universit\u00e9 de Rennes 1; 2014."},{"key":"1280_CR15","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1016\/j.jbi.2017.01.005","volume":"67","author":"LC Roman","year":"2017","unstructured":"Roman LC, Ancker JS, Johnson SB, Senathirajah Y. Navigation in the electronic health record: a review of the safety and usability literature. J Biomed Inform. 2017;67:69\u201379.","journal-title":"J Biomed Inform"},{"key":"1280_CR16","doi-asserted-by":"publisher","first-page":"e191709","DOI":"10.1001\/jamanetworkopen.2019.1709","volume":"2","author":"LM Mazur","year":"2019","unstructured":"Mazur LM, Mosaly PR, Moore C, Marks L. Association of the Usability of electronic health records with cognitive workload and performance levels among physicians. JAMA Netw Open. 2019;2:e191709.","journal-title":"JAMA Netw Open"},{"key":"1280_CR17","doi-asserted-by":"publisher","first-page":"960","DOI":"10.1016\/j.promfg.2018.03.137","volume":"22","author":"H Taherdoost","year":"2018","unstructured":"Taherdoost H. A review of technology acceptance and adoption models and theories. Proc Manufact. 2018;22:960\u20137.","journal-title":"Proc Manufact"},{"key":"1280_CR18","first-page":"298","volume":"257","author":"R Marcilly","year":"2019","unstructured":"Marcilly R, Douze L, Bousquet C, Pelayo S. Modeling keyword search strategy: analysis of Pharmacovigilance specialists\u2019 search of MedDRA terms. Stud Health Technol Inform. 2019;257:298\u2013302.","journal-title":"Stud Health Technol Inform"},{"key":"1280_CR19","doi-asserted-by":"crossref","unstructured":"Ferr\u00e9 S. Sparklis: An expressive query builder for SPARQL endpoints with guidance in natural language. Semantic Web. 2017;8:405-18.","DOI":"10.3233\/SW-150208"},{"key":"1280_CR20","doi-asserted-by":"publisher","DOI":"10.1201\/9781420090512","volume-title":"Foundations of semantic web technologies","author":"P Hitzler","year":"2009","unstructured":"Hitzler P, Kr\u00f6tzsch M, Rudolph S. Foundations of semantic web technologies. Boca Raton: CRC Press; 2009."},{"key":"1280_CR21","volume-title":"Dynamic taxonomies and faceted search: theory, practice, and experience","year":"2009","unstructured":"Sacco GM, Tzitzikas Y, editors. Dynamic taxonomies and faceted search: theory, practice, and experience. Dordrecht: Springer; 2009."},{"key":"1280_CR22","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1145\/1121949.1121979","volume":"49","author":"G Marchionini","year":"2006","unstructured":"Marchionini G. Exploratory search: from finding to understanding. Commun ACM. 2006;49:41.","journal-title":"Commun ACM"},{"key":"1280_CR23","doi-asserted-by":"publisher","first-page":"125","DOI":"10.3233\/SW-2011-0041","volume":"2","author":"V Lopez","year":"2011","unstructured":"Lopez V, Uren V, Sabou M, Motta E. Is question answering fit for the semantic web?: a survey. Semantic Web. 2011;2:125\u201355.","journal-title":"Semantic Web"},{"key":"1280_CR24","volume-title":"Int Conf Semantic Web Applications and Tools for Life Sciences (SWAT4LS)","author":"C Bobed","year":"2018","unstructured":"Bobed C, Douze L, Ferr\u00e9 S, Marcilly R. Sparklis over PEGASE knowledge graph: a new tool for pharmacovigilance. In: Baker AS, Beyan OD, Marshall MS, editors. Int Conf Semantic Web Applications and Tools for Life Sciences (SWAT4LS); 2018."},{"key":"1280_CR25","first-page":"213","volume":"264","author":"J-B Lamy","year":"2019","unstructured":"Lamy J-B, Thuy VB, Lillo-Le Lou\u00ebt A, Bousquet C. An iconic approach to the browsing of medical terminologies. Stud Health Technol Inform. 2019;264:213\u20137.","journal-title":"Stud Health Technol Inform"},{"key":"1280_CR26","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1186\/1472-6947-8-16","volume":"8","author":"J-B Lamy","year":"2008","unstructured":"Lamy J-B, Duclos C, Bar-Hen A, Ouvrard P, Venot A. An iconic language for the graphical representation of medical concepts. BMC Med Inform Decis Mak. 2008;8:16.","journal-title":"BMC Med Inform Decis Mak"},{"key":"1280_CR27","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1016\/j.knosys.2017.08.011","volume":"135","author":"J-B Lamy","year":"2017","unstructured":"Lamy J-B, Soualmia LF. Formalization of the semantics of iconic languages: an ontology-based method and four semantic-powered applications. Knowl-Based Syst. 2017;135:159\u201379.","journal-title":"Knowl-Based Syst"},{"key":"1280_CR28","doi-asserted-by":"publisher","first-page":"741","DOI":"10.1016\/0020-7373(92)90039-N","volume":"36","author":"PG Polson","year":"1992","unstructured":"Polson PG, Lewis C, Rieman J, Wharton C. Cognitive walkthroughs: a method for theory-based evaluation of user interfaces. Int J Man-Machine Studies. 1992;36:741\u201373.","journal-title":"Int J Man-Machine Studies"},{"key":"1280_CR29","doi-asserted-by":"crossref","unstructured":"Lewis C, Wharton C. Cognitive Walkthroughs. In: Handbook of Human-Computer Interaction: Elsevier; 1997. p. 717\u201332. Available from: https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/B9780444818621500960 [cited 2019 Feb 21].","DOI":"10.1016\/B978-044481862-1.50096-0"},{"key":"1280_CR30","volume-title":"Handbook of usability testing: how to plan, design, and conduct effective tests","author":"J Rubin","year":"2008","unstructured":"Rubin J, Chisnell D. Handbook of usability testing: how to plan, design, and conduct effective tests. 2nd ed. Wiley Pub: Indianapolis; 2008.","edition":"2"},{"key":"1280_CR31","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-08-052029-2.50007-3","volume-title":"Usability engineering","author":"J Nielsen","year":"1993","unstructured":"Nielsen J. Usability engineering. Boston: Academic Press; 1993."},{"key":"1280_CR32","doi-asserted-by":"publisher","first-page":"340","DOI":"10.1016\/j.ijmedinf.2008.10.002","volume":"78","author":"MWM Jaspers","year":"2009","unstructured":"Jaspers MWM. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform. 2009;78:340\u201353.","journal-title":"Int J Med Inform"},{"key":"1280_CR33","unstructured":"Peute LW, Driest KF, Marcilly R, Bras Da Costa S, Beuscart-Zephir M-C, Jaspers MWM. A framework for reporting on human factor\/usability studies of health information technologies. Stud Health Technol Inform 2013;194:54\u201360."},{"key":"1280_CR34","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1080\/014492997119806","volume":"16","author":"DL Scapin","year":"1997","unstructured":"Scapin DL, Bastien JMC. Ergonomic criteria for evaluating the ergonomic quality of interactive systems. Behav Inform Technol. 1997;16:220\u201331.","journal-title":"Behav Inform Technol"},{"key":"1280_CR35","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1186\/s13256-018-1580-8","volume":"12","author":"N Belai","year":"2018","unstructured":"Belai N, Gebrehiwet S, Fitsum Y, Russom M. Hydrochlorothiazide and risk of hearing disorder: a case series. J Med Case Rep. 2018;12:135.","journal-title":"J Med Case Rep"},{"key":"1280_CR36","doi-asserted-by":"publisher","first-page":"344","DOI":"10.1016\/j.ijantimicag.2018.04.007","volume":"52","author":"A Lavergne","year":"2018","unstructured":"Lavergne A, Vigneau C, Polard E, Triquet L, Rioux-Leclercq N, Tattevin P, et al. Acute kidney injury during treatment with high-dose cloxacillin: a report of 23 cases and literature review. Int J Antimicrob Agents. 2018;52:344\u20139.","journal-title":"Int J Antimicrob Agents"},{"key":"1280_CR37","doi-asserted-by":"publisher","first-page":"1594","DOI":"10.3201\/eid2408.180460","volume":"24","author":"M Martinot","year":"2018","unstructured":"Martinot M, Ahle G, Petrosyan I, Martinez C, Gorun DM, Mohseni-Zadeh M, et al. Progressive multifocal Leukoencephalopathy after treatment with Nivolumab. Emerg Infect Dis. 2018;24:1594\u20136.","journal-title":"Emerg Infect Dis"},{"key":"1280_CR38","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1186\/s13256-018-1690-3","volume":"12","author":"A Ponard","year":"2018","unstructured":"Ponard A, Ferreira-Maldent N, Ertault M, Delain M, Amraoui K, Regina S, et al. Glycemic dysregulation in a patient with type 2 diabetes treated with 5-azacitidine: a case report. J Med Case Rep. 2018;12:199.","journal-title":"J Med Case Rep"},{"key":"1280_CR39","first-page":"94","volume":"61","author":"S Mukherjee","year":"2019","unstructured":"Mukherjee S, Era N, Mukherjee M, Tripathi SK. Leukocytoclastic vasculitis secondary to clozapine. Indian J Psychiatry. 2019;61:94\u20136.","journal-title":"Indian J Psychiatry"},{"key":"1280_CR40","first-page":"32","volume":"13","author":"CS Lam","year":"2018","unstructured":"Lam CS, Umi Kalthum MN, Norshamsiah MD, Bastion M. Case series of children with steroid-induced glaucoma. Malays Fam Physician. 2018;13:32\u20137.","journal-title":"Malays Fam Physician"},{"key":"1280_CR41","volume-title":"SUS: a \u201cquick and dirty\u201d usability scale. Usability evaluation in industry","author":"J Brooke","year":"1996","unstructured":"Brooke J. SUS: a \u201cquick and dirty\u201d usability scale. Usability evaluation in industry. London: CRC Press; 1996."},{"key":"1280_CR42","volume-title":"Usability inspection methods","year":"1994","unstructured":"Nielsen J, Mack RL, editors. Usability inspection methods. New York: Wiley; 1994."},{"key":"1280_CR43","volume-title":"Human factors engineering: design of medical devices","author":"Association for the Advancement of Medical Instrumentation, American National Standards Institute, Association for the Advancement of Medical Instrumentation","year":"2018","unstructured":"Association for the Advancement of Medical Instrumentation, American National Standards Institute, Association for the Advancement of Medical Instrumentation. Human factors engineering: design of medical devices. 2018."},{"key":"1280_CR44","doi-asserted-by":"publisher","first-page":"574","DOI":"10.1080\/10447310802205776","volume":"24","author":"A Bangor","year":"2008","unstructured":"Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Hum Comput Interact. 2008;24:574\u201394.","journal-title":"Int J Hum Comput Interact"},{"key":"1280_CR45","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1207\/s15327590ijhc0903_2","volume":"9","author":"A Sears","year":"1997","unstructured":"Sears A. Heuristic walkthroughs: finding the problems without the noise. Int J Hum Comput Interact. 1997;9:213\u201334.","journal-title":"Int J Hum Comput Interact"},{"key":"1280_CR46","first-page":"851","volume":"124","author":"E Sundvall","year":"2006","unstructured":"Sundvall E, Nystr\u00f6m M, Petersson H, Ahlfeldt H. Interactive visualization and navigation of complex terminology systems, exemplified by SNOMED CT. Stud Health Technol Inform. 2006;124:851\u20136.","journal-title":"Stud Health Technol Inform"},{"key":"1280_CR47","volume-title":"SNOMED - 5-step briefing. SNOMED International, leading healthcare terminology, worldwide","author":"SNOMED International","year":"2020","unstructured":"SNOMED International. SNOMED - 5-step briefing. SNOMED International, leading healthcare terminology, worldwide. 2020."},{"key":"1280_CR48","doi-asserted-by":"publisher","first-page":"3","DOI":"10.3414\/ME0630","volume":"49","author":"R Khajouei","year":"2010","unstructured":"Khajouei R, Jaspers MWM. The impact of CPOE medication systems\u2019 design aspects on usability, workflow and medication orders: a systematic review. Methods Inf Med. 2010;49:3\u201319.","journal-title":"Methods Inf Med"},{"key":"1280_CR49","doi-asserted-by":"publisher","first-page":"260","DOI":"10.1016\/j.jbi.2015.03.006","volume":"55","author":"R Marcilly","year":"2015","unstructured":"Marcilly R, Ammenwerth E, Vasseur F, Roehrer E, Beuscart-Z\u00e9phir M-C. Usability flaws of medication-related alerting functions: a systematic qualitative review. J Biomed Inform. 2015;55:260\u201371.","journal-title":"J Biomed Inform"},{"key":"1280_CR50","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.jbi.2015.02.006","volume":"55","author":"LWP Peute","year":"2015","unstructured":"Peute LWP, de Keizer NF, Jaspers MWM. The value of retrospective and concurrent think aloud in formative usability testing of a physician data query tool. J Biomed Inform. 2015;55:1\u201310.","journal-title":"J Biomed Inform"},{"key":"1280_CR51","first-page":"137","volume":"37","author":"A Shiri","year":"2013","unstructured":"Shiri A, Ruecker S, Bouchard M, Doll L, Fiorentino C. User Evaluation of Searchling and T-saurus: Multilingual Thesaurus-Enhanced Visual Interfaces for Digital Libraries \/ \u00c9valuation par les usagers de Searchling et de T-saurus : Les interfaces visuelles \u00e0 thesaurus multilingue pour les biblioth\u00e8ques num\u00e9riques. Can J Inf Libr Sci. 2013;37:137\u201360.","journal-title":"Can J Inf Libr Sci"},{"key":"1280_CR52","doi-asserted-by":"publisher","first-page":"835","DOI":"10.1016\/j.compedu.2009.09.012","volume":"54","author":"G Vega-Gorgojo","year":"2010","unstructured":"Vega-Gorgojo G, Bote-Lorenzo ML, Asensio-P\u00e9rez JI, G\u00f3mez-S\u00e1nchez E, Dimitriadis YA, Jorr\u00edn-Abell\u00e1n IM. Semantic search of tools for collaborative learning with the Ontoolsearch system. Comput Educ. 2010;54:835\u201348.","journal-title":"Comput Educ"},{"key":"1280_CR53","doi-asserted-by":"publisher","first-page":"741","DOI":"10.1006\/ijhc.2000.0416","volume":"53","author":"AG Sutcliffe","year":"2000","unstructured":"Sutcliffe AG, Ennis M, Hu J. Evaluating the effectiveness of visual user interfaces for information retrieval. Int J Hum Comput Stud. 2000;53:741\u201363.","journal-title":"Int J Hum Comput Stud"},{"key":"1280_CR54","doi-asserted-by":"publisher","first-page":"748","DOI":"10.1016\/j.compbiomed.2005.04.009","volume":"36","author":"C Henegar","year":"2006","unstructured":"Henegar C, Bousquet C, Lillo-Le Lou\u00ebt A, Degoulet P, Jaulent M-C. Building an ontology of adverse drug reactions for automated signal generation in pharmacovigilance. Comput Biol Med. 2006;36:748\u201367.","journal-title":"Comput Biol Med"},{"key":"1280_CR55","doi-asserted-by":"publisher","first-page":"e97","DOI":"10.1016\/j.ijmedinf.2009.01.001","volume":"78","author":"RK Pearson","year":"2009","unstructured":"Pearson RK, Hauben M, Goldsmith DI, Gould AL, Madigan D, O\u2019Hara DJ, et al. Influence of the MedDRA\u00ae hierarchy on pharmacovigilance data mining results. Int J Med Inform. 2009;78:e97\u2013103.","journal-title":"Int J Med Inform"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-020-01280-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-020-01280-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-020-01280-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,10,10]],"date-time":"2021-10-10T12:26:53Z","timestamp":1633868813000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-020-01280-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,9]]},"references-count":55,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["1280"],"URL":"https:\/\/doi.org\/10.1186\/s12911-020-01280-1","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,9]]},"assertion":[{"value":"28 July 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 September 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 October 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"According to French law, studies in human and social sciences in the field of health do not require approval by an ethics committee (decree n\u00b02017\u2013884 of 9th May 2017, article 2). Thus, in accordance with French law, this study did not require approval from an ethics committee to be conducted. Nevertheless, all procedures involving human participants were performed in accordance with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Witten informed consent was obtained from all individual participants included in the study.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"261"}}