{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T11:31:29Z","timestamp":1771500689814,"version":"3.50.1"},"reference-count":40,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2005,9,22]],"date-time":"2005-09-22T00:00:00Z","timestamp":1127347200000},"content-version":"vor","delay-in-days":630,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc of Assoc for Info"],"published-print":{"date-parts":[[2004,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The goal of this research is to automatically predict human judgments of document qualities such as subjectivity, verbosity and depth. In this paper, we explore the behavior of adjectives as indicators of subjectivity in documents. Specifically, we test whether a subset of automatically derived subjective adjectives (Wiebe, ), selected <jats:italic>a priori<\/jats:italic>, behaves differently than other adjectives. 3,200 documents were ranked by 100 subjects as being high or low in nine document qualities (Tang, Ng, Strzalkowski, &amp; Kantor, ). We report a statistically significant correlation between the occurrence of adjectives in documents and human judgments of subjectivity. More importantly, we find that the subset of subjective adjectives is more strongly correlated with subjectivity than adjectives in general. These results can be used to identify document qualities for use in information retrieval and question\u2010answering systems.<\/jats:p>","DOI":"10.1002\/meet.1450410141","type":"journal-article","created":{"date-parts":[[2005,9,22]],"date-time":"2005-09-22T14:38:14Z","timestamp":1127399894000},"page":"349-359","source":"Crossref","is-referenced-by-count":7,"title":["Adjectives as indicators of subjectivity in documents"],"prefix":"10.1002","volume":"41","author":[{"given":"Robert","family":"Rittman","sequence":"first","affiliation":[]},{"given":"Nina","family":"Wacholder","sequence":"additional","affiliation":[]},{"given":"Paul","family":"Kantor","sequence":"additional","affiliation":[]},{"given":"Kwong Bor","family":"Ng","sequence":"additional","affiliation":[]},{"given":"Tomek","family":"Strzalkowski","sequence":"additional","affiliation":[]},{"given":"Ying","family":"Sun","sequence":"additional","affiliation":[]}],"member":"311","published-online":{"date-parts":[[2005,9,22]]},"reference":[{"key":"e_1_2_8_2_1","unstructured":"ARDA. (2003). Advanced research and development activity.http:\/\/www.ic\u2010arda.org\/"},{"key":"e_1_2_8_3_1","unstructured":"ARDA\/AQUAINT. (2003).Advanced question and answering for intelligence.http:\/\/www.ic\u2010arda.org\/InfoExploit\/aquaint\/"},{"key":"e_1_2_8_4_1","unstructured":"Bai B. Ng K. B. Kantor P. Strzalkowski T. &Sun Y.(in press).The institutional dimension of document qualities judgment."},{"key":"e_1_2_8_5_1","unstructured":"Bouillon P. &Viegas E. (1999).The description of adjectives for natural language processing: Theoretical and applied perspectives. Atelier Thematique TALN Cargese. Retrieved November 19 2003 fromhttp:\/\/citeseer.nj.nec.com\/342843.html"},{"key":"e_1_2_8_6_1","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324999002181"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4573(99)00072-2"},{"key":"e_1_2_8_8_1","unstructured":"Cunningham H. Maynard D. Bontcheva K. Tablan V. &Wilks Y.(2000). Experience of using GATE for NLP R & D. InWorkshop on Using Toolsets and Architectures To Build NLP Systems at COLING\u20102000 Luxembourg."},{"key":"e_1_2_8_9_1","doi-asserted-by":"crossref","unstructured":"Dave K. Lawrence S. &Pennock D. M.(2003).Mining the peanut gallery: Opinion extraction and semantic classification of product reviews. Proceedings of the 12th International World Wide Web Conference Budapest. Retrieved November 4 2003 fromhttp:\/\/citeseer.nj.nec.com\/dave03mining.html","DOI":"10.1145\/775224.775226"},{"key":"e_1_2_8_10_1","doi-asserted-by":"crossref","unstructured":"Day D. Aberdeen J. Hirschman L. Kozierok R. Robinson P. &Vilain M. (1997).Mixed\u2010initiative development of language processing systems. Fifth Conference on Applied Natural Language Processing Association for Computational Linguistics. Retrieved on December 12 2002 fromhttp:\/\/www.mitre.org\/tecrmology\/alembic\u2010workbench\/ANLP97\u2010bigger.html","DOI":"10.3115\/974557.974608"},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199404)45:3<124::AID-ASI2>3.0.CO;2-8"},{"key":"e_1_2_8_12_1","doi-asserted-by":"crossref","unstructured":"Hatzivassiloglou V. &McKeown K.(1997).Predicting the semantic orientation of adjectives. Thirty\u2010Fifth Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics Madrid 174\u2013181. Retrieved November 4 2003 fromhttp:\/\/citeseer.nj.nec.com\/hatzivassiloglou97predicting.html","DOI":"10.3115\/979617.979640"},{"key":"e_1_2_8_13_1","doi-asserted-by":"crossref","unstructured":"Hatzivassiloglou V. &Wiebe J. (2000).Effects of adjective orientation and gradability on sentence subjectivity. Eighteenth International Conference on Computational Linguistics COLING\u20102000. Retrieved November 4 2003 fromhttp:\/\/citeseer.nj.nec.com\/432435.html","DOI":"10.3115\/990820.990864"},{"key":"e_1_2_8_14_1","volume-title":"Applied discriminant analysis","author":"Huberty C. J.","year":"1994"},{"key":"e_1_2_8_15_1","first-page":"1","article-title":"Co\u2010occurrences of antonymous adjectives and their contexts","volume":"17","author":"Justeson J.","year":"1991","journal-title":"Computational Linguistics"},{"key":"e_1_2_8_16_1","first-page":"1","article-title":"Principled disambiguation: Discriminating adjective senses with modified nouns","volume":"21","author":"Justeson J.","year":"1995","journal-title":"Computational Linguistics"},{"key":"e_1_2_8_17_1","unstructured":"Katerattanakul P. &Siau K. (1999).Measuring information quality of web sites: Development of an instrument. Proceeding of the 20th international conference on Information Systems Charlotte NC 279 \u2013 285. Retrieved January 30 2003 fromhttp:\/\/portal.acm.org\/"},{"key":"e_1_2_8_18_1","doi-asserted-by":"crossref","unstructured":"Keast G. Toms E. &Cherry J.(2001).Measuring the reputation of web sites: A preliminary exploration. International Conference on Digital Libraries. Proceedings of the First ACM\/IEEE\u2010CS Joint Conference on Digital libraries Roanoke VA 77\u201378. Retrieved January 27 2003 fromhttp:\/\/portal.acm.org\/","DOI":"10.1145\/379437.379453"},{"key":"e_1_2_8_19_1","doi-asserted-by":"crossref","unstructured":"Lin D.(1998).Automatic retrieval and clustering of similar words. Thirty\u2010Sixth Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics COLING\u2010ACL 1998 Proceedings of the Conference Volume 2 Montreal 768\u2013774. Retrieved January 27 2003 fromhttp:\/\/citeseer.ist.psu.edu\/16998.html","DOI":"10.3115\/980432.980696"},{"key":"e_1_2_8_20_1","first-page":"29","article-title":"What's the real thing? Paradoxes and prototypes of an English adjective","volume":"27","author":"Magnusson U.","year":"2003","journal-title":"ICAME Journal (International Computer Archive of Modern English)"},{"key":"e_1_2_8_21_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199709)48:9<810::AID-ASI6>3.0.CO;2-U"},{"key":"e_1_2_8_22_1","unstructured":"Naumann F. &Rolker C.(2000).Assessment methods for information quality criteria. Proceedings of the International Conference on Information Quality Cambridge MA. Retrieved February 25 2003 fromhttp:\/\/www.hiqiq.de\/publications.html"},{"key":"e_1_2_8_23_1","doi-asserted-by":"crossref","unstructured":"Ng K. B. Kantor P. Tang R. Rittman R. Small S. Song P.et al. (2003).Identification of effective predictive variables for document qualities. Proceedings of the 66th Annual Meeting of the American Society for Information Science and Technology 221\u2013229.","DOI":"10.1002\/meet.1450400128"},{"key":"e_1_2_8_24_1","unstructured":"Ng K. B. Kantor P. Tang R. Strzalkowski T. Bai B. Rittman R. Song P. Sun Y. &Wacholder N.(2004).Personalization of document qualities prediction. Manuscript submitted for publication."},{"key":"e_1_2_8_25_1","volume-title":"Lexical semantics of adjectives: A microtheory of adjectival meaning","author":"Raskin V.","year":"1995"},{"key":"e_1_2_8_26_1","doi-asserted-by":"crossref","unstructured":"Raskin V. &Nirenburg S.(1996).Adjectival modification in text meaning representation. Proceedings of the 16th International Conference on Computational Linguistics Vol. 2: COLING\u20101996 Copenhagen 842\u2013847.","DOI":"10.3115\/993268.993314"},{"key":"e_1_2_8_27_1","unstructured":"Rieh S. Y.(2000).Information quality and cognitive authority in the World Wide Web. Unpublished dissertation. Rutgers the State University of New Jersey."},{"key":"e_1_2_8_28_1","unstructured":"Saracevic T.(1996).Relevance reconsidered. Information science: Integration in perspectives. Proceedings of the Second Conference on Conceptions of Library and Information Science Copenhagen 201\u2013218."},{"key":"e_1_2_8_29_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(1999)50:12<1051::AID-ASI2>3.0.CO;2-Z"},{"key":"e_1_2_8_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(90)90050-C"},{"key":"e_1_2_8_31_1","doi-asserted-by":"crossref","unstructured":"Small S. Liu T. Shimizu N. &Strzalkowski T.(2003).HITIQA: An interactive question answering system: A preliminary report. Proceedings of the ACL 2003 Workshop on Multilingual Summarization and Question Answering 46\u201353.","DOI":"10.3115\/1119312.1119318"},{"key":"e_1_2_8_32_1","doi-asserted-by":"crossref","unstructured":"Staab S. &Hahn U.(1997).Conceptualizing adjectives. Proceedings of the 21st Annual German Conference on Artificial Intelligence (KI\u201097) Freiburg 267\u2013278. Retrieved November 4 2003 fromhttp:\/\/citeseer.nj.nec.com\/staab97conceptualizing.html","DOI":"10.1007\/3540634932_21"},{"key":"e_1_2_8_33_1","unstructured":"Tang R.(1999).Use of relevance criteria across stages of document evaluation: A micro level and macro level analysis. Unpublished dissertation. University of North Carolina at Chapel Hill."},{"key":"e_1_2_8_34_1","doi-asserted-by":"crossref","unstructured":"Tang R. Ng K. B. Strzalkowski T. &Kantor P. (2003).Toward machine understanding of information quality. Proceedings of the 66th Annual Meeting of the American Society for Information Science and Technology 213\u2013220.","DOI":"10.1002\/meet.1450400127"},{"key":"e_1_2_8_35_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.1116"},{"key":"e_1_2_8_36_1","doi-asserted-by":"publisher","DOI":"10.6028\/NIST.SP.500-249"},{"key":"e_1_2_8_37_1","unstructured":"Wiebe J.(2000a).Learning subjective adjectives from corpora. Proceedings of the 17th National Conference on Artificial Intelligence (AAAI\u20102000) Austin. Retrieved December 4 2003 fromhttp:\/\/www.cs.pitt.edu\/\u223cwiebe\/pubs\/aaai00"},{"key":"e_1_2_8_38_1","unstructured":"Wiebe J.(2000b). [Learning subjective adjectives from corpora]. Unpublished list of subjective adjectives. Retrieved November 21 2003 fromhttp:\/\/www.cs.pitt.edu\/\u223cwiebe\/pubs\/aaai00\/adjsMPQA"},{"key":"e_1_2_8_39_1","first-page":"246","volume-title":"Development and use of a gold standard data set for subjectivity classifications","author":"Wiebe J.","year":"1999"},{"key":"e_1_2_8_40_1","unstructured":"Wiebe J. &Wilson T.(2002). [Learning subjective adjectives from corpora]. Unpublished specification for producing the adjective list. Retrieved December 4 2003 fromhttp:\/\/www.cs.pitt.edu\/\u223cwiebe\/pubs\/aaaiOO\/processIDadjs.ps"},{"key":"e_1_2_8_41_1","volume-title":"Learning subjective language. (Technical Report TR\u201002\u2013100)","author":"Wiebe J.","year":"2002"}],"container-title":["Proceedings of the American Society for Information Science and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fmeet.1450410141","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/meet.1450410141","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,20]],"date-time":"2025-10-20T10:53:32Z","timestamp":1760957612000},"score":1,"resource":{"primary":{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/10.1002\/meet.1450410141"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2004,1]]},"references-count":40,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2004,1]]}},"alternative-id":["10.1002\/meet.1450410141"],"URL":"https:\/\/doi.org\/10.1002\/meet.1450410141","archive":["Portico"],"relation":{},"ISSN":["0044-7870","1550-8390"],"issn-type":[{"value":"0044-7870","type":"print"},{"value":"1550-8390","type":"electronic"}],"subject":[],"published":{"date-parts":[[2004,1]]}}}