{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T19:08:58Z","timestamp":1776366538193,"version":"3.51.2"},"reference-count":5,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2016,2,24]],"date-time":"2016-02-24T00:00:00Z","timestamp":1456272000000},"content-version":"vor","delay-in-days":419,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["asistdl.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Proc. Assoc. Info. Sci. Tech."],"published-print":{"date-parts":[[2015,1]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:p>In this paper we propose and test a methodology for evaluation of statements of a multi\u2010viewpoint ontology by crowdsourcing. The task for the workers was to assess each of the given statement as true statements, controversial viewpoint statement or error. Typically, in crowdsourcing experiments the workers are asked for their personal opinions on the given subject. However, in our case their ability to objectively assess others\u2019 opinions is examined as well. We conducted two large\u2010scale crowdsourcing experiments with about 750 ontological statements originating from diverse single\u2010viewpoint ontologies. Our results show substantially higher accuracy in evaluation for the objective assessment approach compared to the experiment based on personal opinions.<\/jats:p>","DOI":"10.1002\/pra2.2015.145052010068","type":"journal-article","created":{"date-parts":[[2016,2,27]],"date-time":"2016-02-27T05:56:50Z","timestamp":1456552610000},"page":"1-4","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Subjective vs. objective evaluation of ontological statements with crowdsourcing"],"prefix":"10.1002","volume":"52","author":[{"given":"Eden S.","family":"Erez","sequence":"first","affiliation":[{"name":"Bar\u2010Ilan University  Ramat\u2010Gan Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maayan","family":"Zhitomirsky\u2010Geffet","sequence":"additional","affiliation":[{"name":"Bar\u2010Ilan University  Ramat\u2010Gan Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Judit","family":"Bar\u2010Ilan","sequence":"additional","affiliation":[{"name":"Bar\u2010Ilan University  Ramat\u2010Gan Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2016,2,24]]},"reference":[{"key":"e_1_2_5_2_1","doi-asserted-by":"crossref","unstructured":"Mortensen J. M. MintyE.P. JanuszukM. et al. (2014).Using the wisdom of the crowds to find critical errors in biomedical ontologies: a study of SNOMED CT.Journal of American Medical Information Association. In press.","DOI":"10.1136\/amiajnl-2014-002901"},{"key":"e_1_2_5_3_1","unstructured":"Mortensen J. M. Musen M. A. andN. F.Noy. (2013).Crowdsourcing the Verification of Relationships in Biomedical Ontologies.AMIA 2013 Annual Symposium."},{"key":"e_1_2_5_4_1","doi-asserted-by":"crossref","unstructured":"Noy N. F. Mortensen J. M. Alexander P. R. andM. A.Musen(2013) Mechanical Turk as an Ontology Engineer? Using Microtasks as a Component of an Ontology\u2010Engineering Workflow InProceedings of the Web Science 2013 Conference Paris.","DOI":"10.1145\/2464464.2464482"},{"key":"e_1_2_5_5_1","doi-asserted-by":"publisher","DOI":"10.1108\/OIR-03-2014-0073"},{"key":"e_1_2_5_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-22630-4_6"}],"container-title":["Proceedings of the Association for Information Science and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fpra2.2015.145052010068","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fpra2.2015.145052010068","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/pra2.2015.145052010068","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/pra2.2015.145052010068","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/pra2.2015.145052010068","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T18:02:36Z","timestamp":1761069756000},"score":1,"resource":{"primary":{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/10.1002\/pra2.2015.145052010068"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,1]]},"references-count":5,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2015,1]]}},"alternative-id":["10.1002\/pra2.2015.145052010068"],"URL":"https:\/\/doi.org\/10.1002\/pra2.2015.145052010068","archive":["Portico"],"relation":{},"ISSN":["2373-9231","2373-9231"],"issn-type":[{"value":"2373-9231","type":"print"},{"value":"2373-9231","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,1]]},"assertion":[{"value":"2016-02-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}