{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,30]],"date-time":"2026-01-30T06:23:39Z","timestamp":1769754219652,"version":"3.49.0"},"reference-count":34,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2016,2,24]],"date-time":"2016-02-24T00:00:00Z","timestamp":1456272000000},"content-version":"vor","delay-in-days":419,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["asistdl.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Proc. Assoc. Info. Sci. Tech."],"published-print":{"date-parts":[[2015,1]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:p>To make phenotypic characters of organisms widely useful for computerized biology research, biocurators manually convert character descriptions to a structured format, for example the Entity\u2010Quality (EQ) format. The manual approach is time consuming and affected by inter\u2010curator variations. In this paper we report a software application, CharaParser+EQ, to our knowledge the first software that produces EQ statements from textual character descriptions. We report a recent experiment that evaluates the performance of the software against three experienced biocurators. While the software is still far from being able to compete with biocurators on this highly intellectual task, the results show (1) CharaParser+EQ's performance (precision and recall) is greatly improved compared to a previous version, (2) the completeness of the ontologies used in the process has significant impact both on the software's EQ generation performance and on the agreement among curators, and (3) unlimited access to external knowledge (published papers, books) by curators has no significant impact on inter\u2010curator agreements. A detailed error analysis that compares machine and curator generated EQs is included.<\/jats:p>","DOI":"10.1002\/pra2.2015.145052010020","type":"journal-article","created":{"date-parts":[[2016,2,27]],"date-time":"2016-02-27T05:56:50Z","timestamp":1456552610000},"page":"1-10","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["CharaParser+EQ: Performance evaluation without gold standard"],"prefix":"10.1002","volume":"52","author":[{"given":"Hong","family":"Cui","sequence":"first","affiliation":[{"name":"School of Information Resources and Library Science University of Arizona  Tucson AZ 85719"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wasila","family":"Dahdul","sequence":"additional","affiliation":[{"name":"Department of Biology University of South Dakota  Vermillion SD 57069"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexander T.","family":"Dececchi","sequence":"additional","affiliation":[{"name":"Department of Biology University of South Dakota  Vermillion SD 57069"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nizar","family":"Ibrahim","sequence":"additional","affiliation":[{"name":"Department of Organismal Biology and Anatomy  Chicago IL 60637"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paula","family":"Mabee","sequence":"additional","affiliation":[{"name":"Department of Biology University of South Dakota  Vermillion SD 57069"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James P.","family":"Balhoff","sequence":"additional","affiliation":[{"name":"National Evolutionary Synthesis Center  Durham NC 27705"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hariharan","family":"Gopalakrishnan","sequence":"additional","affiliation":[{"name":"School of Information Resources and Library Science University of Arizona  Tucson AZ 85719"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2016,2,24]]},"reference":[{"key":"e_1_2_11_2_1","unstructured":"Apache UIMA Development Community.Apache ConceptMapper annotator user Guide(2011). Retrieved July 6 2015 fromhttps:\/\/uima.apache.org\/d\/uima-addons-current\/ConceptMapper\/ConceptMapperAnnotatorUserGuide.html."},{"key":"e_1_2_11_3_1","doi-asserted-by":"publisher","DOI":"10.1093\/database\/bas056"},{"key":"e_1_2_11_4_1","unstructured":"Aronson A.R(2001).Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. InProceedings of American Medical Informatics Association 2001 Annual Symposium(pp.17\u201321)."},{"key":"e_1_2_11_5_1","doi-asserted-by":"publisher","DOI":"10.1136\/jamia.2009.002733"},{"key":"e_1_2_11_6_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0010500"},{"key":"e_1_2_11_7_1","unstructured":"Banerjee S.&Lavie A.(2005).METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. InProceedings of Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and\/or Summarization at the 43rd Annual Meeting of the Association of Computational Linguistics (ACL\u20102005). Retrieved July 6 2015 fromhttp:\/\/www.cs.cmu.edu\/~alavie\/papers\/BanerjeeLavie2005-final.pdf."},{"key":"e_1_2_11_8_1","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkn886"},{"key":"e_1_2_11_9_1","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkm826"},{"key":"e_1_2_11_10_1","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkq1077"},{"key":"e_1_2_11_11_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-6-S1-S17"},{"key":"e_1_2_11_12_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.22618"},{"key":"e_1_2_11_13_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.21246"},{"key":"e_1_2_11_14_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1002854"},{"key":"e_1_2_11_15_1","doi-asserted-by":"publisher","DOI":"10.1186\/2041-1480-5-34"},{"key":"e_1_2_11_16_1","article-title":"An efficient solution for mapping free text to ontology terms","volume":"21","author":"Dai M.","year":"2008","journal-title":"AMIA Summit on Translational Bioinformatics"},{"key":"e_1_2_11_17_1","unstructured":"Dececchi T.A. Balhoff J.P. Lapp H.& et al. (in press).Toward synthesizing our knowledge of morphology: Using ontologies and machine reasoning to extract presence\/absence evolutionary phenotypes across studies.Systematic Biology."},{"key":"e_1_2_11_18_1","doi-asserted-by":"publisher","DOI":"10.1197\/jamia.M1176"},{"key":"e_1_2_11_19_1","unstructured":"Francis W.N.&Kucera H.(1964).Manual of information to accompany a standard corpus of present\u2010day edited American English for use with digital computers. Retrieved July 6 2015 fromhttp:\/\/icame.uib.no\/brown\/bcm.html"},{"key":"e_1_2_11_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ecoinf.2012.09.003"},{"key":"e_1_2_11_21_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-6-S1-S1"},{"key":"e_1_2_11_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-374814-0.00017-3"},{"key":"e_1_2_11_23_1","doi-asserted-by":"publisher","DOI":"10.3115\/1075096.1075150"},{"key":"e_1_2_11_24_1","doi-asserted-by":"publisher","DOI":"10.1186\/gb-2008-9-s2-s1"},{"key":"e_1_2_11_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCBB.2010.61"},{"key":"e_1_2_11_26_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1439-0426.2012.01985.x"},{"key":"e_1_2_11_27_1","unstructured":"Maynard D.(2005).Benchmarking ontology\u2010based annotation tools for the semantic web. InProceedings of 6th International Conference on Language Resources and Evaluation (LREC). Retrieved July 6 2015 fromhttps:\/\/gate.ac.uk\/sale\/lrec2008\/benchmarking.pdf."},{"key":"e_1_2_11_28_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-9-327"},{"key":"e_1_2_11_29_1","doi-asserted-by":"publisher","DOI":"10.1186\/gb-2012-13-1-r5"},{"key":"e_1_2_11_30_1","unstructured":"PATO: Phenotypic Quality Ontology. Retrieved July 6 2015 fromhttp:\/\/wiki.obofoundry.org\/wiki\/index.php\/PATO:Main_Page."},{"key":"e_1_2_11_31_1","doi-asserted-by":"publisher","DOI":"10.1017\/S135132490400347X"},{"key":"e_1_2_11_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcss.2011.10.010"},{"key":"e_1_2_11_33_1","doi-asserted-by":"publisher","DOI":"10.4202\/app.2010.0101"},{"key":"e_1_2_11_34_1","doi-asserted-by":"crossref","unstructured":"Soo V\u2010W. YangS\u2010Y. Chen S\u2010L. et al. (2004).Ontology acquisition and semantic retrieval from semantic annotated Chinese poetry. InProceedings of 2004 Joint ACM\/IEEE Conference on Digital Libraries(pp.345\u2010346).","DOI":"10.1145\/996350.996429"},{"key":"e_1_2_11_35_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-10-326"}],"container-title":["Proceedings of the Association for Information Science and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fpra2.2015.145052010020","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fpra2.2015.145052010020","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/pra2.2015.145052010020","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/pra2.2015.145052010020","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/pra2.2015.145052010020","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T18:04:23Z","timestamp":1761069863000},"score":1,"resource":{"primary":{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/10.1002\/pra2.2015.145052010020"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,1]]},"references-count":34,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2015,1]]}},"alternative-id":["10.1002\/pra2.2015.145052010020"],"URL":"https:\/\/doi.org\/10.1002\/pra2.2015.145052010020","archive":["Portico"],"relation":{},"ISSN":["2373-9231","2373-9231"],"issn-type":[{"value":"2373-9231","type":"print"},{"value":"2373-9231","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,1]]},"assertion":[{"value":"2016-02-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}