{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,29]],"date-time":"2025-09-29T11:53:21Z","timestamp":1759146801427,"version":"3.41.2"},"reference-count":38,"publisher":"Emerald","issue":"3","license":[{"start":{"date-parts":[[2014,5,6]],"date-time":"2014-05-06T00:00:00Z","timestamp":1399334400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2014,5,6]]},"abstract":"<jats:sec>\n               <jats:title content-type=\"abstract-heading\">Purpose<\/jats:title>\n               <jats:p> \u2013 The purpose of this paper is to examine the effect of the Helping Interdisciplinary Vocabulary Engineering (HIVE) system on the inter-indexer consistency of information professionals when assigning keywords to a scientific abstract. This study examined first, the inter-indexer consistency of potential HIVE users; second, the impact HIVE had on consistency; and third, challenges associated with using HIVE. <\/jats:p>\n            <\/jats:sec>\n            <jats:sec>\n               <jats:title content-type=\"abstract-heading\">Design\/methodology\/approach<\/jats:title>\n               <jats:p> \u2013 A within-subjects quasi-experimental research design was used for this study. Data were collected using a task-scenario based questionnaire. Analysis was performed on consistency results using Hooper's and Rolling's inter-indexer consistency measures. A series of <jats:italic>t<\/jats:italic>-tests was used to judge the significance between consistency measure results. <\/jats:p>\n            <\/jats:sec>\n            <jats:sec>\n               <jats:title content-type=\"abstract-heading\">Findings<\/jats:title>\n               <jats:p> \u2013 Results suggest that HIVE improves inter-indexing consistency. Working with HIVE increased consistency rates by 22 percent (Rolling's) and 25 percent (Hooper's) when selecting relevant terms from all vocabularies. A statistically significant difference exists between the assignment of free-text keywords and machine-aided keywords. Issues with homographs, disambiguation, vocabulary choice, and document structure were all identified as potential challenges. <\/jats:p>\n            <\/jats:sec>\n            <jats:sec>\n               <jats:title content-type=\"abstract-heading\">Research limitations\/implications<\/jats:title>\n               <jats:p> \u2013 Research limitations for this study can be found in the small number of vocabularies used for the study. Future research will include implementing HIVE into the Dryad Repository and studying its application in a repository system. <\/jats:p>\n            <\/jats:sec>\n            <jats:sec>\n               <jats:title content-type=\"abstract-heading\">Originality\/value<\/jats:title>\n               <jats:p> \u2013 This paper showcases several features used in HIVE system. By using traditional consistency measures to evaluate a semantic web technology, this paper emphasizes the link between traditional indexing and next generation machine-aided indexing (MAI) tools.<\/jats:p>\n            <\/jats:sec>","DOI":"10.1108\/jd-07-2012-0083","type":"journal-article","created":{"date-parts":[[2014,6,17]],"date-time":"2014-06-17T23:40:45Z","timestamp":1403048445000},"page":"307-329","source":"Crossref","is-referenced-by-count":4,"title":["HIVEing: the effect of a semantic web technology on inter-indexer consistency"],"prefix":"10.1108","volume":"70","author":[{"given":"Hollie","family":"White","sequence":"first","affiliation":[]},{"given":"Craig","family":"Willis","sequence":"additional","affiliation":[]},{"given":"Jane","family":"Greenberg","sequence":"additional","affiliation":[]}],"member":"140","reference":[{"key":"key2020061111403118100_b1","doi-asserted-by":"crossref","unstructured":"Anderson, J.D.\n                and \n                  Perez-Carballo, J.\n                (2001), \u201cThe nature of indexing: how humans and machines analyze messages and text for retrieval. Part I: research, and the nature of human indexing\u201d, Information Processing and Management, Vol. 37 No. 2, pp. 231-254.","DOI":"10.1016\/S0306-4573(00)00026-1"},{"key":"key2020061111403118100_b2","doi-asserted-by":"crossref","unstructured":"Braam, R.R.\n                and \n                  Bruil, J.\n                (1992), \u201cQuality of indexing information: authors\u2019 views on indexing of their articles in chemical abstracts online CA-file\u201d, Journal of Information Science, Vol. 18 No. 5, pp. 399-408.","DOI":"10.1177\/016555159201800508"},{"key":"key2020061111403118100_b3","unstructured":"Bruce, T.R.\n                and \n                  Hillman, D.I.\n                (2004), \u201cThe continuum of metadata quality: defining, expressing, and exploiting\u201d, in \n                  Hillman, D.I.\n                and \n                  Westbrooks, E.L.\n                (Eds), Metadata in Practice, ALA Editions, Chicago, IL, pp. 238-256."},{"key":"key2020061111403118100_b4","unstructured":"Chan, L.M.\n               , \n                  Richmond, P.A.\n                and \n                  Svenonius, E.\n                (Eds) (1985), Theory of Subject Analysis: a Sourcebook, Libraries Unlimited, Littleton Colorodo, CO."},{"key":"key2020061111403118100_b5","unstructured":"Chen, X.\n                (2008), \u201cThe influence of existing consistency measures on the relationship between indexing consistency and exhaustivity\u201d, 10th International ISKO (International Society of Knowledge Organization) Conference. Montreal, Canada 5-8 August, Ergon Verlag, Wurzburg."},{"key":"key2020061111403118100_b6","unstructured":"Cleverdon, C.W.\n                (1970), The Effect of Variations in Relevance Assessments in Comparative Experimental tests of Index Languages, Cranfield Institute of Technology, Cranfield."},{"key":"key2020061111403118100_b7","doi-asserted-by":"crossref","unstructured":"Greenberg, J.\n               , \n                  Losee, R.\n               , \n                  P\u00e9rez Ag\u00fcera, J.R.\n               , \n                  Scherle, R.\n               , \n                  White, H.\n                and \n                  Willis, C.\n                (2011), \u201cHIVE: helping interdisciplinary vocabulary engineering\u201d, Bulletin of the American Society for Information Science and Technology, Vol. 37 No. 4, pp. 23-26.","DOI":"10.1002\/bult.2011.1720370407"},{"key":"key2020061111403118100_b8","doi-asserted-by":"crossref","unstructured":"Field, B.J.\n                (1975), \u201cTowards automatic indexing: automatic assignment of controlled-language indexing from free indexing\u201d, Journal of Documentation, Vol. 31 No. 4, pp. 246-265.","DOI":"10.1108\/eb026605"},{"key":"key2020061111403118100_b10","unstructured":"Frank, E.T.\n               , \n                  Paynter, G.W.\n               , \n                  Witten, I.H.\n               , \n                  Gutwin, C.A.\n                and \n                  Nevill-Manning, C.G.\n                (1999), \u201cDomain specific keyphrase extraction\u201d, Sixteenth International Joint Conference on Artificial Intelligence. Stockholm, 31 July-6 August, Morgan Kaufmann Publishers, San Francisco, CA, available at: http:\/\/ijcai.org\/Past%20Proceedings\/IJCAI-99%20VOL-2\/PDF\/002.pdf"},{"key":"key2020061111403118100_b11","unstructured":"Hlava, M.M.K.\n                (2005), \u201cAutomatic indexing\u201d, Information Outlook, Vol. 9 No. 8, pp. 22-23."},{"key":"key2020061111403118100_b12","unstructured":"Hodge, G.M.\n                and \n                  Milstead, J.L.\n                (1998), Computer Support to Indexing, National Federation of Abstracting and Information Services, Philadelphia, PA."},{"key":"key2020061111403118100_b13","unstructured":"Hooper, R.S.\n                (1969), Indexer Consistency Tests: Origin, Measurement, Results, and Utilization, IBM Corporation, Bethesda, MD."},{"key":"key2020061111403118100_b14","doi-asserted-by":"crossref","unstructured":"Hughes, A.V.\n                and \n                  Rafferty, P.\n                (2011), \u201cInter-indexer consistency in graphic materials indexing at the national library of Wales\u201d, Journal of Documentation, Vol. 67 No. 1, pp. 9-32.","DOI":"10.1108\/00220411111105434"},{"key":"key2020061111403118100_b15","unstructured":"International Standards Office\n                (1985), ISO 5963 \u2013 Documentation \u2013 Methods for Examining Documents, Determining their Subjects, and Selecting Indexing Terms, ISO, Geneva."},{"key":"key2020061111403118100_b16","unstructured":"Kim, W.\n               , \n                  Aronson, A.R.\n                and \n                  Wilbur, W.J.\n                (2001), \u201cAutomatic MeSH term assignment and quality assessment\u201d, in \n                  Bakken, S.\n                (Ed.), AMIA (American Medical Informatics Association) Annual Symposium. Washington, DC, 3-7 November, AMIA, Bethesda, MD, pp. 319-323."},{"key":"key2020061111403118100_b17","unstructured":"Lancaster, F.W.\n                (1986), Vocabulary Control for Information Retrieval, IRP, Arlington, VA."},{"key":"key2020061111403118100_b18","unstructured":"Lancaster, F.W.\n                (2003), Indexing and Abstracting in Theory and Practice, 3rd ed., Facet Publishing, London."},{"key":"key2020061111403118100_b19","unstructured":"Leonard, L.E.\n                (1977), Inter-Indexer Consistency Studies, 1954-1975: a Review of the Literature and Summary of Study Results, University of Illinois Graduate School of Library Science, Urbana, IL."},{"key":"key2020061111403118100_b20","unstructured":"Mai, J.-E.\n                (2000), \u201cThe subject indexing process: an investigation of problems in knowledge representation\u201d, PhD, University of Texas, Austin."},{"key":"key2020061111403118100_b21","unstructured":"Markey, K.\n                (1984), \u201cInterindexer consistency tests: a literature review and report of a test of consistency in indexing visual materials\u201d, Library and Information Science Research, Vol. 6 No. 2, pp. 155-177."},{"key":"key2020061111403118100_b22","doi-asserted-by":"crossref","unstructured":"Medelyan, O.\n                and \n                  Witten, I.H.\n                (2006), \u201cMeasuring inter-indexer consistency using a Thesaurus\u201d, JCDL (Joint Conference on Digital Libraries) 2006 Opening Information Horizons, Chapel Hill, NC, 11-15 June, Association of Computing Machinery, New York, NY, available at: http:\/\/researchcommons.waikato.ac.nz\/bitstream\/handle\/10289\/1345\/06-OM-IHW-Thesaurus-auto-keyphrase.pdf?sequence=1","DOI":"10.1145\/1141753.1141816"},{"key":"key2020061111403118100_b23","doi-asserted-by":"crossref","unstructured":"Medelyan, O.\n                and \n                  Witten, I.H.\n                (2008), \u201cDomain independent automatic keyphrase indexing with small training sets\u201d, Journal of American Society for Information Science and Technology, Vol. 59 No. 7, pp. 1026-1040.","DOI":"10.1002\/asi.20790"},{"key":"key2020061111403118100_b24","unstructured":"Medelyan, O.\n                (2009), Human-Competitive Automatic Topic Indexing, University of Waikato, Hamilton, available at: http:\/\/researchcommons.waikato.ac.nz\/handle\/10289\/3513 (accessed January 30, 2013)."},{"key":"key2020061111403118100_b25","unstructured":"Moens, M.F.\n                (2000), Automatic Indexing and Abstracting of Text Documents, Kluwer, Boston, MA."},{"key":"key2020061111403118100_b26","doi-asserted-by":"crossref","unstructured":"Park, J.-R.\n                (2009), \u201cMetadata quality in digital repositories: a survey of the current state of the art\u201d, Cataloging and Classification Quarterly, Vol. 47 Nos 3-4, pp. 213-228.","DOI":"10.1080\/01639370902737240"},{"key":"key2020061111403118100_b27","doi-asserted-by":"crossref","unstructured":"Reich, P.\n                and \n                  Biever, E.J.\n                (1991), \u201cIndexing consistency: the input\/output function of Thesauri\u201d, College and Research Libraries, Vol. 52 No. 4, pp. 336-342.","DOI":"10.5860\/crl_52_04_336"},{"key":"key2020061111403118100_b28","doi-asserted-by":"crossref","unstructured":"Rolling, L.\n                (1981), \u201cIndexing consistency, quality, and efficiency\u201d, Information Processing and Management, Vol. 17 No. 2, pp. 69-76.","DOI":"10.1016\/0306-4573(81)90028-5"},{"key":"key2020061111403118100_b29","unstructured":"Salton, G.\n                (1975), Dynamic Information and Library Processing, Prentice Hall, Englewood Cliffs, NJ."},{"key":"key2020061111403118100_b30","doi-asserted-by":"crossref","unstructured":"Sebastiani, F.\n                (2002), \u201cMachine learning in automated text categorization\u201d, ACM Computing Surveys, Vol. 34 No. 1, pp. 1-47.","DOI":"10.1145\/505282.505283"},{"key":"key2020061111403118100_b31","doi-asserted-by":"crossref","unstructured":"Sievert, M.C.\n                and \n                  Andrews, M.J.\n                (1991), \u201cIndexing consistency in information science abstracts\u201d, Journal of the American Society for Information Science, Vol. 41 No. 1, pp. 1-6.","DOI":"10.1002\/(SICI)1097-4571(199101)42:1<1::AID-ASI1>3.0.CO;2-9"},{"key":"key2020061111403118100_b32","doi-asserted-by":"crossref","unstructured":"Silvester, J.P.\n               , \n                  Genuardi, M.T.\n                and \n                  Klingbeil, P.H.\n                (1994), \u201cMachine-aided indexing at NASA\u201d, Information Processing and Management, Vol. 30 No. 5, pp. 631-645.","DOI":"10.1016\/0306-4573(94)90075-2"},{"key":"key2020061111403118100_b34","unstructured":"Sparck Jones, K.\n                (1971), Automatic Keyword Classification, Archon, London."},{"key":"key2020061111403118100_b35","doi-asserted-by":"crossref","unstructured":"Svenonius, E.\n                (1986), \u201cUnanswered questions in the design of controlled vocabularies\u201d, Journal of the American Society for Information Science, Vol. 37 No. 5, pp. 331-340.","DOI":"10.1002\/(SICI)1097-4571(198609)37:5<331::AID-ASI8>3.0.CO;2-E"},{"key":"key2020061111403118100_b36","unstructured":"Wolfram, D.\n                and \n                  Olsen, H.\n                (2007), \u201cA method for comparing large scale inter-indexer consistency using IR modeling\u201d, Proceedings of the 35th Annual Conference of the Canadian Association for Information Science, Montreal, May 12-20, available at: www.cais-acsi.ca\/proceedings\/2007\/wolfram_2007.pdf"},{"key":"key2020061111403118100_b37","doi-asserted-by":"crossref","unstructured":"Veldus-Stokolov, N.\n                (1982), \u201cOn automatic support to indexing a life sciences database\u201d, Information Processing and Management, Vol. 18 No. 6, pp. 313-321.","DOI":"10.1016\/0306-4573(82)90018-8"},{"key":"key2020061111403118100_frd1","unstructured":"Fidel, R.\n                (1992), \u201cWho needs a controlled vocabulary?\u201d, Special Libraries, Vol. 83 No. 1, pp. 1-9."},{"key":"key2020061111403118100_frd2","doi-asserted-by":"crossref","unstructured":"Slamencka, V.\n                and \n                  Jacoby, J.\n                (1963), \u201cEffect of indexing aides on the reliability of indexers\u201d, Documentation Incorporated, Betheseda, MD, available at: www.dtic.mil\/cgi-bin\/GetTRDoc?AD=AD0410032 (accessed August 9, 2011).","DOI":"10.21236\/AD0410032"},{"key":"key2020061111403118100_frd3","doi-asserted-by":"crossref","unstructured":"Zunde, P.\n                and \n                  Dexter, M.E.\n                (1969), \u201cIndexing consistency and quality\u201d, American Documentation, Vol. 20 No. 3, pp. 259-267.","DOI":"10.1002\/asi.4630200313"}],"container-title":["Journal of Documentation"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.emeraldinsight.com\/doi\/full-xml\/10.1108\/JD-07-2012-0083","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/JD-07-2012-0083\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/JD-07-2012-0083\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T22:34:24Z","timestamp":1753396464000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/jd\/article\/70\/3\/307-329\/202410"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,5,6]]},"references-count":38,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2014,5,6]]}},"alternative-id":["10.1108\/JD-07-2012-0083"],"URL":"https:\/\/doi.org\/10.1108\/jd-07-2012-0083","relation":{},"ISSN":["0022-0418"],"issn-type":[{"type":"print","value":"0022-0418"}],"subject":[],"published":{"date-parts":[[2014,5,6]]}}}