{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T19:19:49Z","timestamp":1778959189124,"version":"3.51.4"},"reference-count":6,"publisher":"MIT Press - Journals","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computational Linguistics"],"published-print":{"date-parts":[[2000,9]]},"abstract":"<jats:p> In a medical information extraction system, we use common word association techniques to extract side-effect-related terms. Many of these terms have a frequency of less than five. Standard word-association-based applications disregard the lowest-frequency words, and hence disregard useful information. We therefore devised an extraction system for the full word frequency range. This system computes the significance of association by the log-likelihood ratio and Fisher's exact test. The output of the system shows a recurrent, corpus-independent pattern in both recall and the number of significant words. We will explain these patterns by the statistical behavior of the lowest-frequency words. We used Dutch verb-particle combinations as a second and independent collocation extraction application to illustrate the generality of the observed phenomena. We will conclude that a) word-association-based extraction systems can be enhanced by also considering the lowest-frequency words, b) significance levels should not be fixed but adjusted for the optimal window size, c) hapax legomena, words occurring only once, should be disregarded a priori in the statistical analysis, and d) the distribution of the targets to extract should be considered in combination with the extraction method. <\/jats:p>","DOI":"10.1162\/089120100561719","type":"journal-article","created":{"date-parts":[[2002,7,27]],"date-time":"2002-07-27T11:56:30Z","timestamp":1027770990000},"page":"301-317","source":"Crossref","is-referenced-by-count":19,"title":["Extracting the Lowest-Frequency Words: Pitfalls and Possibilities"],"prefix":"10.1162","volume":"26","author":[{"given":"Marc","family":"Weeber","sequence":"first","affiliation":[{"name":"University of Groningen, Groningen University Institute for Drug Exploration, Department of Social Pharmacy and Pharmacoepidemiology, Ant. Deusinglaan 1, 9713 AV Groningen, The Netherlands."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rein","family":"Vos","sequence":"additional","affiliation":[{"name":"University of Groningen, University of Maastricht, Faculty of Health Sciences, Department of Health Ethics and Philosophy, P.O. Box 616, 6200 MD, Maastricht, The Netherlands."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"R. Harald","family":"Baayen","sequence":"additional","affiliation":[{"name":"Max Planck Institute for Psycholinguistics, Max Planck Institute for Psycholinguistics, P.O. Box 310, 6500 AH Nijmegen."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"issue":"1","key":"p_2","first-page":"22","volume":"16","author":"Church Kenneth W","year":"1990","journal-title":"Computational Linguistics"},{"key":"p_3","doi-asserted-by":"publisher","DOI":"10.1145\/168173.168412"},{"issue":"1","key":"p_4","first-page":"61","volume":"19","author":"Dunning Ted","year":"1993","journal-title":"Computational Linguistics"},{"key":"p_6","doi-asserted-by":"publisher","DOI":"10.1145\/6497.214326"},{"key":"p_10","doi-asserted-by":"publisher","DOI":"10.1007\/BF01870611"},{"issue":"1","key":"p_11","first-page":"143","volume":"19","author":"Smadja Frank","year":"1993","journal-title":"Computational Linguistics"}],"container-title":["Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/089120100561719","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:41:41Z","timestamp":1615585301000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/coli\/article\/26\/3\/301-317\/1654"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2000,9]]},"references-count":6,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2000,9]]}},"alternative-id":["10.1162\/089120100561719"],"URL":"https:\/\/doi.org\/10.1162\/089120100561719","relation":{},"ISSN":["0891-2017","1530-9312"],"issn-type":[{"value":"0891-2017","type":"print"},{"value":"1530-9312","type":"electronic"}],"subject":[],"published":{"date-parts":[[2000,9]]}}}