{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,11]],"date-time":"2026-06-11T07:35:55Z","timestamp":1781163355177,"version":"3.54.1"},"reference-count":51,"publisher":"MIT Press","issue":"2","license":[{"start":{"date-parts":[[2021,2,22]],"date-time":"2021-02-22T00:00:00Z","timestamp":1613952000000},"content-version":"vor","delay-in-days":52,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,7,15]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Previous studies of research methods in Library and Information Science (LIS) lack consensus in how to define or classify research methods, and there have been no studies on automated recognition of research methods in the scientific literature of this field. This work begins to fill these gaps by studying how the scope of \u201cresearch methods\u201d in LIS has evolved, and the challenges in automatically identifying the usage of research methods in LIS literature. We collected 2,599 research articles from three LIS journals. Using a combination of content analysis and text mining methods, a sample of this collection is coded into 29 different concepts of research methods and is then used to test a rule-based automated method for identifying research methods reported in the scientific literature. We show that the LIS field is characterized by the use of an increasingly diverse range of methods, many of which originate outside the conventional boundaries of LIS. This implies increasing complexity in research methodology and suggests the need for a new approach towards classifying LIS research methods to capture the complex structure and relationships between different aspects of methods. Our automated method is the first of its kind in LIS, and sets an important reference for future research.<\/jats:p>","DOI":"10.1162\/qss_a_00123","type":"journal-article","created":{"date-parts":[[2021,2,10]],"date-time":"2021-02-10T16:27:55Z","timestamp":1612974475000},"page":"698-732","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":19,"title":["Towards automated analysis of research methods in library and information science"],"prefix":"10.1162","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8587-8618","authenticated-orcid":true,"given":"Ziqi","family":"Zhang","sequence":"first","affiliation":[{"name":"Information School, The University of Sheffield"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9978-3428","authenticated-orcid":true,"given":"Winnie","family":"Tam","sequence":"additional","affiliation":[{"name":"University of Manchester Library, The University of Manchester"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2587-245X","authenticated-orcid":true,"given":"Andrew","family":"Cox","sequence":"additional","affiliation":[{"name":"Information School, The University of Sheffield"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2021,7,15]]},"reference":[{"key":"2021072016160736700_bib1","first-page":"113","article-title":"Are cited references meaningful? Measuring semantic relatedness in citation analysis","volume-title":"BIRNDL@SIGIR","author":"Alam","year":"2017"},{"key":"2021072016160736700_bib2","doi-asserted-by":"crossref","first-page":"546","DOI":"10.18653\/v1\/S17-2091","article-title":"SemEval 2017 Task 10: ScienceIE\u2014Extracting Keyphrases and Relations from Scientific Publications","volume-title":"Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)","author":"Augenstein","year":"2017"},{"issue":"3","key":"2021072016160736700_bib3","first-page":"1","article-title":"In search of research methods used in information science","volume":"18","author":"Bernhard","year":"1993","journal-title":"Canadian Journal of Information and Library Science"},{"issue":"1\/2","key":"2021072016160736700_bib4","first-page":"1","article-title":"Since Shaughnessy: Research methods in library and information science dissertation, 1975\u20131989","volume":"19","author":"Blake","year":"1994","journal-title":"Collection Development"},{"key":"2021072016160736700_bib5","volume-title":"Experimental and quasi-experimental designs for research","author":"Campbell","year":"1966"},{"issue":"2","key":"2021072016160736700_bib6","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1002\/asi.22767","article-title":"A text mining approach to assist the general public in the retrieval of legal documents","volume":"64","author":"Chen","year":"2013","journal-title":"Journal of the American Society for Information Science and Technology"},{"key":"2021072016160736700_bib7","first-page":"827","article-title":"Rule-based information extraction is dead! Long live rule-based information extraction systems!","volume-title":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing","author":"Chiticariu","year":"2013"},{"key":"2021072016160736700_bib8","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1016\/j.lisr.2014.09.003","article-title":"Research methods in library and information science: A content analysis","volume":"37","author":"Chu","year":"2015","journal-title":"Library and Information Science Research"},{"key":"2021072016160736700_bib9","first-page":"215","article-title":"FLUX-CiM: Fexible unsupervised extraction of citation metadata","volume-title":"Proceedings of the 7th ACM\/IEEE Joint Conference on Digital Libraries","author":"Cortez","year":"2007"},{"key":"2021072016160736700_bib10","first-page":"1604","article-title":"Rhetorical move detection in English abstracts: Multi-label sentence classifiers and their annotated corpora","volume-title":"Proceedings of the Eighth International Conference on Language Resources and Evaluation","author":"Dayrell","year":"2012"},{"issue":"3","key":"2021072016160736700_bib11","first-page":"550","article-title":"A readability level prediction tool for K\u201312 books","volume":"67","author":"Denning","year":"2015","journal-title":"Journal of the American Society for Information Science and Technology"},{"key":"2021072016160736700_bib12","first-page":"1","article-title":"A typology of research methods within the social sciences","author":"Durrant","year":"2004","journal-title":"NCRM Working Paper Series"},{"issue":"1","key":"2021072016160736700_bib13","first-page":"741","article-title":"Research methods and techniques in Spanish library and information science journals (2012\u20132014)","volume":"22","author":"Ferran-Ferrer","year":"2017","journal-title":"Information Research"},{"key":"2021072016160736700_bib14","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1016\/j.lisr.2008.04.001","article-title":"Are we there yet?: Mixed methods research in library and information science","volume":"30","author":"Fidel","year":"2008","journal-title":"Library & Information Science Research"},{"issue":"5","key":"2021072016160736700_bib15","doi-asserted-by":"crossref","first-page":"974","DOI":"10.1108\/JD-04-2017-0058","article-title":"A knowledge lens for information literacy: Conceptual framework and case study","volume":"73","author":"Freeburg","year":"2017","journal-title":"Journal of Documentation"},{"issue":"1","key":"2021072016160736700_bib16","doi-asserted-by":"crossref","first-page":"101003","DOI":"10.1016\/j.lisr.2020.101003","article-title":"Mixed methods research in library and information science: A methodological review","volume":"42","author":"Grankikov","year":"2020","journal-title":"Library & Information Science Research"},{"key":"2021072016160736700_bib17","first-page":"1","article-title":"Analyzing the dynamics of research by extracting key aspects of scientific papers","volume-title":"Proceedings of the 5th International Joint Conference on Natural Language Processing","author":"Gupta","year":"2011"},{"issue":"10","key":"2021072016160736700_bib18","doi-asserted-by":"crossref","first-page":"421","DOI":"10.4156\/jcit.vol7.issue10.50","article-title":"An automated cue word based text extraction","volume":"7","author":"Hanyurwimfura","year":"2012","journal-title":"Journal of Convergence Information Technology"},{"key":"2021072016160736700_bib19","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1016\/j.lisr.2007.11.007","article-title":"Empirical research methods reported in high-profile LIS journal literature","volume":"30","author":"Hider","year":"2008","journal-title":"Library & Information Science Research"},{"key":"2021072016160736700_bib20","first-page":"381","article-title":"Identifying sections in scientific abstracts using conditional random fields","volume-title":"Proceedings of the Third International Joint Conference on Natural Language Processing","author":"Hirohata","year":"2008"},{"key":"2021072016160736700_bib21","first-page":"1211","article-title":"Method mention extraction from scientific research paper","volume-title":"Proceedings of COLING: Technical paper","author":"Houngb","year":"2012"},{"key":"2021072016160736700_bib22","first-page":"395","article-title":"Content analysis of research articles in library and information science","volume":"12","author":"J\u00e4rvelin","year":"1990","journal-title":"Library & Information Science Research"},{"issue":"4","key":"2021072016160736700_bib23","doi-asserted-by":"crossref","first-page":"376","DOI":"10.2307\/40324247","article-title":"Research record","volume":"37","author":"Kim","year":"1996","journal-title":"Journal of Education for Library and Information Science"},{"issue":"11","key":"2021072016160736700_bib24","first-page":"8","article-title":"Towards semantometrics: A new semantic similarity based measure for assessing a research publications contribution","volume":"20","author":"Knoth","year":"2014","journal-title":"D-Lib Magazine"},{"issue":"2","key":"2021072016160736700_bib25","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.csl.2011.09.001","article-title":"Mining methodologies from NLP publications: A case study in automatic terminology recognition","volume":"26","author":"Kova\u010devi\u0107","year":"2012","journal-title":"Computer Speech & Language"},{"key":"2021072016160736700_bib26","volume-title":"Netnography: Doing ethnographic research online","author":"Kozinets","year":"2010"},{"issue":"1","key":"2021072016160736700_bib27","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1515\/libr.1991.41.1.59","article-title":"Library and information science research in 1975","volume":"41","author":"Kumpulainen","year":"1991","journal-title":"Libri"},{"issue":"7","key":"2021072016160736700_bib28","doi-asserted-by":"crossref","first-page":"991","DOI":"10.1093\/bioinformatics\/bts071","article-title":"Automatic recognition of conceptualization zones in scientific articles and two life science applications","volume":"28","author":"Liakata","year":"2012","journal-title":"Bioinformatics"},{"key":"2021072016160736700_bib29","first-page":"90","article-title":"Extracting formulaic and free text clinical research articles metadata using conditional random fields","volume-title":"Proceedings of the NAACL HLT 2010 Second Louhi Workshop on Text and Data Mining of Health Documents","author":"Lin","year":"2010"},{"key":"2021072016160736700_bib30","article-title":"Review of the typology of research methods within the social sciences","author":"Luff","year":"2015","journal-title":"National Centre For Research Methods Report"},{"issue":"11","key":"2021072016160736700_bib31","doi-asserted-by":"crossref","first-page":"2243","DOI":"10.1002\/asi.21390","article-title":"Participatory design of a health informatics system for rural health practitioners and disadvantaged women","volume":"61","author":"Lustria","year":"2010","journal-title":"Journal of the American Society for Information Science and Technology"},{"key":"2021072016160736700_bib32","doi-asserted-by":"crossref","first-page":"1931","DOI":"10.1007\/s11192-018-2921-5","article-title":"Information extraction from scientific articles: A survey","volume":"117","author":"Nasar","year":"2018","journal-title":"Scientometrics"},{"issue":"6","key":"2021072016160736700_bib33","doi-asserted-by":"crossref","first-page":"066133","DOI":"10.1103\/PhysRevE.69.066133","article-title":"Fast algorithm for detecting community structure in networks","volume":"69","author":"Newman","year":"2004","journal-title":"Physical Review E"},{"issue":"2","key":"2021072016160736700_bib34","doi-asserted-by":"crossref","first-page":"026113","DOI":"10.1103\/PhysRevE.69.026113","article-title":"Finding and evaluating community structure in networks","volume":"69","author":"Newman","year":"2004","journal-title":"Physical Review E"},{"key":"2021072016160736700_bib35","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1016\/j.lisr.2004.04.009","article-title":"The study of research methods in LIS education: Issues in Korean and U.S. universities","volume":"26","author":"Park","year":"2004","journal-title":"Library & Information Science Research"},{"key":"2021072016160736700_bib36","first-page":"1862","article-title":"The ACL RD-TEC 2.0: A language resource for evaluating term extraction and entity recognition methods","volume-title":"Proceedings of the Tenth International Conference on Language Resources and Evaluation","author":"QasemiZadeh","year":"2016"},{"issue":"1\/2","key":"2021072016160736700_bib37","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1108\/NLW-08-2015-0055","article-title":"Research methods used in library and information science during the 1970\u20132010","volume":"117","author":"Risso","year":"2016","journal-title":"New Library World"},{"key":"2021072016160736700_bib38","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1007\/978-3-319-24282-8_18","article-title":"Dr. Inventor framework: Extracting structured information from scientific publications","volume-title":"Discovery Science","author":"Ronzano","year":"2015"},{"issue":"2","key":"2021072016160736700_bib39","doi-asserted-by":"crossref","first-page":"304","DOI":"10.1108\/00220410910937633","article-title":"A study of the use of self-organising maps in information retrieval","volume":"65","author":"Saarikoski","year":"2009","journal-title":"Journal of Documentation"},{"key":"2021072016160736700_bib40","first-page":"37","article-title":"Learning hidden Markov model structure for information extraction","volume-title":"Proceedings of the AAAI\u201999 Workshop Machine Learning for Information Extraction","author":"Seymore","year":"1999"},{"issue":"8","key":"2021072016160736700_bib41","doi-asserted-by":"crossref","first-page":"1485","DOI":"10.1002\/asi.22673","article-title":"A study of open access journals using article processing charges","volume":"63","author":"Solomon","year":"2012","journal-title":"Journal of the American Society for Information Science and Technology"},{"key":"2021072016160736700_bib43","first-page":"3836","article-title":"Typed entity and relation annotation computer science papers","volume-title":"Proceedings of the Tenth International Conference on Language Resources and Evaluation","author":"Tateisi","year":"2016"},{"issue":"4","key":"2021072016160736700_bib44","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1162\/089120102762671936","article-title":"Summarizing scientific articles: Experiments with relevance and rhetorical status","volume":"28","author":"Teufel","year":"2002","journal-title":"Computational Linguistics"},{"key":"2021072016160736700_bib45","doi-asserted-by":"crossref","first-page":"43","DOI":"10.5772\/intechopen.68749","article-title":"Research methods in library and information science","volume-title":"Qualitative versus Quantitative Research","author":"Togia","year":"2017"},{"issue":"2","key":"2021072016160736700_bib46","doi-asserted-by":"crossref","first-page":"523","DOI":"10.1007\/s11192-009-0146-3","article-title":"Software survey: VOSviewer, a computer program for bibliometric mapping","volume":"84","author":"Van Eck","year":"2010","journal-title":"Scientometrics"},{"key":"2021072016160736700_bib47","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1007\/978-3-319-10377-8_13","article-title":"Visualizing bibliometric networks","volume-title":"Measuring scholarly impact: Methods and practice","author":"Van Eck","year":"2014"},{"key":"2021072016160736700_bib48","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1016\/j.lisr.2016.04.002","article-title":"How reference and information service is studied: Research approaches and methods","volume":"38","author":"VanScoy","year":"2016","journal-title":"Library & Information Science Research"},{"issue":"2","key":"2021072016160736700_bib49","doi-asserted-by":"crossref","first-page":"240","DOI":"10.1002\/asi.20987","article-title":"A new approach for detecting scientific specialties from raw cocitation networks","volume":"60","author":"Wallace","year":"2008","journal-title":"Journal of the American Society for Information Science and Technology"},{"issue":"2","key":"2021072016160736700_bib50","doi-asserted-by":"crossref","first-page":"721","DOI":"10.1007\/s11192-018-2768-9","article-title":"Three new bibliometric indicators\/approaches derived from keyword analysis","volume":"116","author":"Wang","year":"2018","journal-title":"Scientometrics"},{"key":"2021072016160736700_bib51","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780199240531.001.0001","volume-title":"The intellectual and social organization of the sciences","author":"Whitley","year":"2000"},{"issue":"11","key":"2021072016160736700_bib52","first-page":"2248","article-title":"A machine-learning approach to coding book reviews as quality indicators: Toward a theory of megacitation","volume":"65","author":"Zuccala","year":"2014","journal-title":"Journal of the American Society for Information Science and Technology"}],"container-title":["Quantitative Science Studies"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/direct.mit.edu\/qss\/article-pdf\/2\/2\/698\/1930792\/qss_a_00123.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/direct.mit.edu\/qss\/article-pdf\/2\/2\/698\/1930792\/qss_a_00123.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,24]],"date-time":"2024-08-24T00:26:35Z","timestamp":1724459195000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/qss\/article\/2\/2\/698\/96161\/Towards-automated-analysis-of-research-methods-in"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021]]},"references-count":51,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,7,15]]}},"URL":"https:\/\/doi.org\/10.1162\/qss_a_00123","relation":{},"ISSN":["2641-3337"],"issn-type":[{"value":"2641-3337","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021]]},"published":{"date-parts":[[2021]]}}}