{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T18:02:23Z","timestamp":1772906543245,"version":"3.50.1"},"reference-count":72,"publisher":"MIT Press","issue":"2","license":[{"start":{"date-parts":[[2024,1,8]],"date-time":"2024-01-08T00:00:00Z","timestamp":1704672000000},"content-version":"vor","delay-in-days":7,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Extracting and identifying latent topics in large text corpora have gained increasing importance in Natural Language Processing (NLP). Most models, whether probabilistic models similar to Latent Dirichlet Allocation (LDA) or neural topic models, follow the same underlying approach of topic interpretability and topic extraction. We propose a method that incorporates a deeper understanding of both sentence and document themes, and goes beyond simply analyzing word frequencies in the data. Through simple corpus expansion, our model can detect latent topics that may include uncommon words or neologisms, as well as words not present in the documents themselves. Additionally, we propose several new evaluation metrics based on intruder words and similarity measures in the semantic space. We present correlation coefficients with human identification of intruder words and achieve near-human level results at the word-intrusion task. We demonstrate the competitive performance of our method with a large benchmark study, and achieve superior results compared with state-of-the-art topic modeling and document clustering models. The code is available at the following link: https:\/\/github.com\/AnFreTh\/STREAM.<\/jats:p>","DOI":"10.1162\/coli_a_00506","type":"journal-article","created":{"date-parts":[[2024,1,8]],"date-time":"2024-01-08T21:19:55Z","timestamp":1704748795000},"page":"619-655","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":10,"title":["Topics in the Haystack: Enhancing Topic Quality through Corpus\n                    Expansion"],"prefix":"10.1162","volume":"50","author":[{"given":"Anton","family":"Thielmann","sequence":"first","affiliation":[{"name":"Institute of Mathematics Clausthal University of Technology. Anton.thielmann@tu-clausthal.de"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Arik","family":"Reuter","sequence":"additional","affiliation":[{"name":"Institute of Mathematics Clausthal University of Technology. Arik.reuter@gmx.de"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Quentin","family":"Seifert","sequence":"additional","affiliation":[{"name":"Spatial Data Science and Statistical Learning University of G\u00f6ttingen. Quentinedward.seifert@uni-goettingen.de"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elisabeth","family":"Bergherr","sequence":"additional","affiliation":[{"name":"Spatial Data Science and Statistical Learning University of G\u00f6ttingen. Elisabeth.bergherr@uni-goettingen.de"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Benjamin","family":"S\u00e4fken","sequence":"additional","affiliation":[{"name":"Institute of Mathematics Clausthal University of Technology. Benjamin.saefken@tu-clausthal.de"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2024,6,1]]},"reference":[{"key":"2024070814302988900_bib1","article-title":"Improving contextualized topic models with negative\n                        sampling","author":"Adhya","year":"2023","journal-title":"arXiv preprint\n                    arXiv:2303.14951"},{"key":"2024070814302988900_bib2","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1145\/1718487.1718499","article-title":"FLDA: Matrix factorization through latent Dirichlet\n                        allocation","volume-title":"Proceedings of the Third ACM\n                        International Conference on Web Search and Data Mining","author":"Agarwal","year":"2010"},{"key":"2024070814302988900_bib3","doi-asserted-by":"publisher","first-page":"420","DOI":"10.1007\/3-540-44503-X_27","article-title":"On the surprising behavior of distance metrics in high\n                        dimensional space","volume-title":"International Conference on\n                        Database Theory","author":"Aggarwal","year":"2001"},{"key":"2024070814302988900_bib4","first-page":"13","article-title":"Evaluating topic coherence using\n                        distributional semantics","volume-title":"Proceedings of the 10th\n                        International Conference on Computational Semantics (IWCS 2013)\u2013Long\n                        Papers","author":"Aletras","year":"2013"},{"key":"2024070814302988900_bib5","article-title":"Top2vec: Distributed representations of\n                        topics","author":"Angelov","year":"2020","journal-title":"arXiv preprint\n                    arXiv:2008.09470"},{"key":"2024070814302988900_bib6","doi-asserted-by":"publisher","first-page":"745","DOI":"10.1109\/ICCONS.2017.8250563","article-title":"An overview of topic modeling methods and\n                        tools","volume-title":"2017 International Conference on\n                        Intelligent Computing and Control Systems (ICICCS)","author":"Barde","year":"2017"},{"issue":"2","key":"2024070814302988900_bib7","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1108\/eb026788","article-title":"Bibliographic classification theory and\n                        text linguistics: Aboutness analysis, intertextuality and the cognitive act\n                        of classifying documents","volume":"42","author":"Beghtol","year":"1986","journal-title":"Journal of\n                        Documentation"},{"key":"2024070814302988900_bib8","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/K17-1022","article-title":"An automatic approach for document-level\n                        topic model evaluation","author":"Bhatia","year":"2017","journal-title":"arXiv preprint\n                        arXiv:1706.05140"},{"key":"2024070814302988900_bib9","doi-asserted-by":"publisher","first-page":"844","DOI":"10.18653\/v1\/D18-1098","article-title":"Topic intrusion for automatic topic model\n                        evaluation","volume-title":"Proceedings of the 2018 Conference on\n                        Empirical Methods in Natural Language Processing","author":"Bhatia","year":"2018"},{"key":"2024070814302988900_bib10","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-short.96","article-title":"Pre-training is a hot topic: Contextualized document\n                        embeddings improve topic coherence","author":"Bianchi","year":"2020","journal-title":"arXiv preprint\n                        arXiv:2004.03974"},{"key":"2024070814302988900_bib11","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1016\/j.ins.2017.02.007","article-title":"A general framework to expand short text for\n                        topic modeling","volume":"393","author":"Bicalho","year":"2017","journal-title":"Information Sciences"},{"key":"2024070814302988900_bib12","volume-title":"Natural Language Processing with Python: Analyzing\n                        Text with the Natural Language Toolkit","author":"Bird","year":"2009"},{"issue":"2","key":"2024070814302988900_bib13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/1667053.1667056","article-title":"The nested Chinese restaurant process and\n                        Bayesian nonparametric inference of topic hierarchies","volume":"57","author":"Blei","year":"2010","journal-title":"Journal of the ACM (JACM)"},{"issue":"1","key":"2024070814302988900_bib14","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1214\/07-AOAS114","article-title":"A correlated topic model of\n                        science","volume":"1","author":"Blei","year":"2007","journal-title":"The Annals of Applied Statistics"},{"issue":"Jan","key":"2024070814302988900_bib15","first-page":"993","article-title":"Latent Dirichlet\n                        allocation","volume":"3","author":"Blei","year":"2003","journal-title":"Journal of Machine Learning\n                        Research"},{"key":"2024070814302988900_bib16","article-title":"Reading tea leaves: How humans interpret topic\n                        models","volume":"22","author":"Chang","year":"2009","journal-title":"Advances in Neural Information Processing\n                        Systems"},{"key":"2024070814302988900_bib17","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1016\/j.neucom.2017.08.029","article-title":"Latent Dirichlet mixture model","volume":"278","author":"Chien","year":"2018","journal-title":"Neurocomputing"},{"key":"2024070814302988900_bib18","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.565","article-title":"All that\u2019s \u2018human\u2019 is not\n                        gold: Evaluating human evaluation of generated text","author":"Clark","year":"2021","journal-title":"arXiv preprint arXiv:2107.00061"},{"key":"2024070814302988900_bib19","doi-asserted-by":"publisher","first-page":"795","DOI":"10.3115\/v1\/P15-1077","article-title":"Gaussian LDA for topic models with word\n                        embeddings","volume-title":"Proceedings of the 53rd Annual\n                        Meeting of the Association for Computational Linguistics and the 7th\n                        International Joint Conference on Natural Language Processing (Volume 1:\n                        Long Papers)","author":"Das","year":"2015"},{"key":"2024070814302988900_bib20","unstructured":"Davison,\n                                Alice\n          .\n                        1982. A systematic definition of sentence\n                        topic. Center for the Study of Reading Technical\n                        Report; no. 264."},{"issue":"4","key":"2024070814302988900_bib21","doi-asserted-by":"publisher","first-page":"797","DOI":"10.1353\/lan.1984.0012","article-title":"Syntactic markedness and the definition of\n                        sentence topic","volume":"60","author":"Davison","year":"1984","journal-title":"Language"},{"key":"2024070814302988900_bib22","article-title":"BERT: Pre-training of deep bidirectional\n                        transformers for language understanding","author":"Devlin","year":"2018","journal-title":"arXiv\n                        preprint arXiv:1810.04805"},{"key":"2024070814302988900_bib23","doi-asserted-by":"publisher","first-page":"439","DOI":"10.1162\/tacl_a_00325","article-title":"Topic modeling in embedding spaces","volume":"8","author":"Dieng","year":"2020","journal-title":"Transactions of the Association for Computational\n                        Linguistics"},{"key":"2024070814302988900_bib24","doi-asserted-by":"publisher","first-page":"1057","DOI":"10.1145\/2911451.2914729","article-title":"Using word embedding to evaluate the coherence\n                        of topics from Twitter data","volume-title":"Proceedings of the\n                        39th International ACM SIGIR Conference on Research and Development in\n                        Information Retrieval","author":"Fang","year":"2016"},{"key":"2024070814302988900_bib25","article-title":"BERTopic: Neural topic modeling with a\n                        class-based tf-idf procedure","author":"Grootendorst","year":"2022","journal-title":"arXiv preprint\n                        arXiv:2203.05794"},{"key":"2024070814302988900_bib26","first-page":"97","article-title":"Towards a definition and hierarchization\n                        of topic","author":"Guijarro","year":"2000","journal-title":"Talk and Text: Studies on Spoken and\n                        Written Discourse"},{"issue":"7302","key":"2024070814302988900_bib27","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1038\/466029a","article-title":"Most people are not weird","volume":"466","author":"Henrich","year":"2010","journal-title":"Nature"},{"issue":"1","key":"2024070814302988900_bib28","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1023\/A:1007617005950","article-title":"Unsupervised learning by probabilistic\n                        latent semantic analysis","volume":"42","author":"Hofmann","year":"2001","journal-title":"Machine Learning"},{"key":"2024070814302988900_bib29","article-title":"Is automated topic model evaluation\n                        broken? The incoherence of coherence","volume":"34","author":"Hoyle","year":"2021","journal-title":"Advances in\n                        Neural Information Processing Systems"},{"key":"2024070814302988900_bib30","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.findings-emnlp.390","article-title":"Are neural topic models\n                        broken?","author":"Hoyle","year":"2022","journal-title":"arXiv preprint arXiv:2210.16162"},{"issue":"54","key":"2024070814302988900_bib31","doi-asserted-by":"publisher","DOI":"10.21105\/joss.02507","article-title":"TTLocVis: A Twitter topic location\n                        visualization package.","volume":"5","author":"Kant","year":"2020","journal-title":"Journal of Open Source\n                        Software"},{"issue":"4","key":"2024070814302988900_bib32","doi-asserted-by":"publisher","first-page":"345","DOI":"10.3758\/BF03198274","article-title":"Initial mention as a signal to thematic\n                        content in technical passages","volume":"8","author":"Kieras","year":"1980","journal-title":"Memory &\n                        Cognition"},{"issue":"6","key":"2024070814302988900_bib33","doi-asserted-by":"publisher","first-page":"541","DOI":"10.3758\/BF03202348","article-title":"Topicalization effects in cued recall of\n                        technical prose","volume":"9","author":"Kieras","year":"1981","journal-title":"Memory & Cognition"},{"key":"2024070814302988900_bib34","doi-asserted-by":"publisher","first-page":"439","DOI":"10.1007\/978-3-319-54395-6_53","article-title":"Questionnaire design","volume-title":"The Palgrave Handbook of Survey Research","author":"Krosnick","year":"2018"},{"key":"2024070814302988900_bib35","article-title":"Correlated topic models","volume":"18","author":"Lafferty","year":"2005","journal-title":"Advances in\n                        Neural Information Processing Systems"},{"key":"2024070814302988900_bib36","article-title":"A neural autoregressive topic\n                        model","volume":"25","author":"Larochelle","year":"2012","journal-title":"Advances in Neural Information Processing\n                        Systems"},{"key":"2024070814302988900_bib37","doi-asserted-by":"publisher","first-page":"530","DOI":"10.3115\/v1\/E14-1056","article-title":"Machine reading tea leaves: Automatically\n                        evaluating topic coherence and topic model quality","volume-title":"Proceedings of the 14th Conference of the European Chapter of the\n                        Association for Computational Linguistics","author":"Lau","year":"2014"},{"key":"2024070814302988900_bib38","first-page":"1188","article-title":"Distributed representations of sentences\n                        and documents","volume-title":"International Conference on\n                        Machine Learning","author":"Le","year":"2014"},{"issue":"6","key":"2024070814302988900_bib39","doi-asserted-by":"publisher","first-page":"690","DOI":"10.1177\/0146167292186005","article-title":"The focus of judgment effect: A question wording effect due\n                        to hypothesis confirnation bias","volume":"18","author":"Lehman","year":"1992","journal-title":"Personality and\n                        Social Psychology Bulletin"},{"key":"2024070814302988900_bib40","doi-asserted-by":"publisher","DOI":"10.24432\/C52G6M","article-title":"Reuters-21578 text categorization collection\n                        data set","author":"Lewis","year":"1997"},{"key":"2024070814302988900_bib41","article-title":"RoBERTa: A robustly optimized BERT\n                        pretraining approach","author":"Liu","year":"2019","journal-title":"arXiv preprint\n                        arXiv:1907.11692"},{"key":"2024070814302988900_bib42","article-title":"Community-detection via hashtag-graphs for\n                        semi-supervised NMF topic models","author":"Luber","year":"2021","journal-title":"arXiv preprint\n                        arXiv:2111.10401"},{"key":"2024070814302988900_bib43","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1076","article-title":"Automatic evaluation of local topic\n                        quality","author":"Lund","year":"2019","journal-title":"arXiv preprint arXiv:1905.13126"},{"key":"2024070814302988900_bib44","first-page":"111","article-title":"More efficient topic modelling through a\n                        noun only approach","volume-title":"Proceedings of the\n                        Australasian Language Technology Association Workshop 2015","author":"Martin","year":"2015"},{"key":"2024070814302988900_bib45","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2020\/4728095","article-title":"A gamma-poisson mixture topic model for\n                        short text","author":"Mazarura","year":"2020","journal-title":"Mathematical Problems in\n                        Engineering"},{"issue":"11","key":"2024070814302988900_bib46","doi-asserted-by":"publisher","DOI":"10.21105\/joss.00205","article-title":"HDBSCAN: Hierarchical density based\n                        clustering","volume":"2","author":"McInnes","year":"2017","journal-title":"The Journal of Open Source\n                        Software"},{"key":"2024070814302988900_bib47","doi-asserted-by":"publisher","DOI":"10.21105\/joss.00861","article-title":"UMAP: Uniform manifold approximation and\n                        projection for dimension reduction","author":"McInnes","year":"2018","journal-title":"arXiv preprint\n                        arXiv:1802.03426"},{"key":"2024070814302988900_bib48","doi-asserted-by":"publisher","first-page":"889","DOI":"10.1145\/2484028.2484166","article-title":"Improving LDA topic models for microblogs via tweet pooling\n                        and automatic labeling","volume-title":"Proceedings of the 36th\n                        International ACM SIGIR Conference on Research and Development in\n                        Information Retrieval","author":"Mehrotra","year":"2013"},{"issue":"3","key":"2024070814302988900_bib49","doi-asserted-by":"publisher","first-page":"102921","DOI":"10.1016\/j.ipm.2022.102921","article-title":"Comparing PSO-based clustering over contextual\n                        vector embeddings to modern topic modeling","volume":"59","author":"Miles","year":"2022","journal-title":"Information Processing & Management"},{"key":"2024070814302988900_bib50","first-page":"100","article-title":"Automatic evaluation of topic\n                        coherence","volume-title":"Human Language Technologies: The 2010\n                        Annual Conference of the North American Chapter of the Association for\n                        Computational Linguistics","author":"Newman","year":"2010"},{"key":"2024070814302988900_bib51","article-title":"Contextualized topic coherence\n                        metrics","author":"Rahimi","year":"2023","journal-title":"arXiv preprint\n                    arXiv:2305.14587"},{"key":"2024070814302988900_bib52","doi-asserted-by":"publisher","first-page":"248","DOI":"10.3115\/1699510.1699543","article-title":"Labeled LDA: A supervised topic model for\n                        credit attribution in multi-labeled corpora","volume-title":"Proceedings of the 2009 Conference on Empirical Methods in Natural\n                        Language Processing","author":"Ramage","year":"2009"},{"key":"2024070814302988900_bib53","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1410","article-title":"Sentence-BERT: Sentence embeddings using\n                        siamese BERT-networks","author":"Reimers","year":"2019","journal-title":"arXiv preprint\n                        arXiv:1908.10084"},{"key":"2024070814302988900_bib54","doi-asserted-by":"publisher","first-page":"659","DOI":"10.1007\/978-0-387-73003-5_196","article-title":"Gaussian mixture\n                        models.","volume":"741","author":"Reynolds","year":"2009","journal-title":"Encyclopedia of Biometrics"},{"key":"2024070814302988900_bib55","article-title":"The author-topic model for authors and\n                        documents","author":"Rosen-Zvi","year":"2012","journal-title":"arXiv preprint\n                    arXiv:1207.4169"},{"key":"2024070814302988900_bib56","article-title":"Automatic Text Processing: The\n                        Transformation, Analysis, and Retrieval of Information by\n                        Computer","author":"Salton","year":"1989"},{"key":"2024070814302988900_bib57","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.135","article-title":"Tired of topic models? Clusters of\n                        pretrained word embeddings make for fast and good topics\n                        too!","author":"Sia","year":"2020","journal-title":"arXiv preprint arXiv:2004.14914"},{"key":"2024070814302988900_bib58","first-page":"16857","article-title":"Mpnet: Masked and permuted pre-training for language\n                        understanding","volume":"33","author":"Song","year":"2020","journal-title":"Advances in Neural Information\n                        Processing Systems"},{"key":"2024070814302988900_bib59","article-title":"Autoencoding variational inference for\n                        topic models","author":"Srivastava","year":"2017","journal-title":"arXiv preprint\n                        arXiv:1703.01488"},{"key":"2024070814302988900_bib60","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.581","article-title":"Re-visiting automated topic model evaluation with large\n                        language models","author":"Stammbach","year":"2023","journal-title":"arXiv preprint\n                        arXiv:2305.12152"},{"key":"2024070814302988900_bib61","doi-asserted-by":"publisher","first-page":"263","DOI":"10.18653\/v1\/2021.eacl-demos.31","article-title":"OCTIS: Comparing and optimizing topic\n                        models is simple!","volume-title":"Proceedings of the 16th\n                        Conference of the European Chapter of the Association for Computational\n                        Linguistics: System Demonstrations","author":"Terragni","year":"2021"},{"key":"2024070814302988900_bib62","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1007\/978-3-030-80599-9_4","article-title":"Word embedding-based topic similarity\n                        measures","volume-title":"International Conference on\n                        Applications of Natural Language to Information Systems","author":"Terragni","year":"2021"},{"key":"2024070814302988900_bib63","doi-asserted-by":"publisher","first-page":"263","DOI":"10.1007\/978-3-030-76620-7_23","article-title":"One-class support vector machine and LDA topic\n                        model integration\u2014evidence for AI patents","volume-title":"Soft Computing: Biomedical and Related Applications","author":"Thielmann","year":"2021"},{"key":"2024070814302988900_bib64","doi-asserted-by":"publisher","first-page":"574","DOI":"10.1080\/02664763.2021.1919063","article-title":"Unsupervised document classification\n                        integrating web scraping, one-class SVM and LDA topic\n                        modelling","author":"Thielmann","year":"2021","journal-title":"Journal of Applied Statistics"},{"key":"2024070814302988900_bib65","article-title":"Human in the loop: How to effectively\n                        create coherent topics by manually labeling only a few documents per\n                        class","author":"Thielmann","year":"2022","journal-title":"arXiv preprint\n                    arXiv:2212.09422"},{"key":"2024070814302988900_bib66","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.372","article-title":"All bark and no bite: Rogue dimensions in\n                        transformer language models obscure representational\n                    quality","author":"Timkey","year":"2021","journal-title":"arXiv preprint arXiv:2109.04404"},{"key":"2024070814302988900_bib67","doi-asserted-by":"publisher","first-page":"101582","DOI":"10.1016\/j.is.2020.101582","article-title":"A review of topic modeling\n                        methods","volume":"94","author":"Vayansky","year":"2020","journal-title":"Information Systems"},{"issue":"6","key":"2024070814302988900_bib68","doi-asserted-by":"publisher","first-page":"102098","DOI":"10.1016\/j.ipm.2019.102098","article-title":"ATM: Adversarial-neural topic model","volume":"56","author":"Wang","year":"2019","journal-title":"Information Processing & Management"},{"key":"2024070814302988900_bib69","first-page":"5776","article-title":"MiniLM: Deep self-attention distillation for task-agnostic\n                        compression of pre-trained transformers","volume":"33","author":"Wang","year":"2020","journal-title":"Advances in\n                        Neural Information Processing Systems"},{"issue":"2","key":"2024070814302988900_bib70","doi-asserted-by":"publisher","first-page":"647","DOI":"10.1007\/s00180-022-01246-z","article-title":"Pseudo-document simulation for comparing\n                        LDA, GSDMM and GPM topic models on short and sparse text using Twitter\n                        data","volume":"38","author":"Weisser","year":"2023","journal-title":"Computational Statistics"},{"issue":"1","key":"2024070814302988900_bib71","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1177\/016555159201800106","article-title":"The automatic identification of stop\n                        words","volume":"18","author":"Wilbur","year":"1992","journal-title":"Journal of Information Science"},{"key":"2024070814302988900_bib72","doi-asserted-by":"publisher","first-page":"2444","DOI":"10.1016\/j.neucom.2017.11.019","article-title":"Corpus-based topic diffusion for short text\n                        clustering","volume":"275","author":"Zheng","year":"2018","journal-title":"Neurocomputing"}],"container-title":["Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/coli\/article-pdf\/50\/2\/619\/2457449\/coli_a_00506.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/coli\/article-pdf\/50\/2\/619\/2457449\/coli_a_00506.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T14:31:06Z","timestamp":1720449066000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/coli\/article\/50\/2\/619\/118990\/Topics-in-the-Haystack-Enhancing-Topic-Quality"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":72,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,6,1]]},"published-print":{"date-parts":[[2024,6,1]]}},"URL":"https:\/\/doi.org\/10.1162\/coli_a_00506","relation":{},"ISSN":["0891-2017","1530-9312"],"issn-type":[{"value":"0891-2017","type":"print"},{"value":"1530-9312","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}