{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,2]],"date-time":"2025-08-02T19:27:05Z","timestamp":1754162825718,"version":"3.41.2"},"reference-count":41,"publisher":"MIT Press","issue":"2","license":[{"start":{"date-parts":[[2021,2,17]],"date-time":"2021-02-17T00:00:00Z","timestamp":1613520000000},"content-version":"vor","delay-in-days":413,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The overall scope of this study is an attempt at a methodological framework for matching publication lists at the national level against a combined set of blacklists for questionable publishing. Using the total verified set of publications from Swedish Higher Education Institutions (HEI) as a case, we examined the number, distribution, and proportion of publishing in questionable journals at the national level. Journal publication data was extracted from the national SwePub database and matched against three curated blacklists of questionable publishing. For the period 2012\u20132017, we identified 1,743 published papers in blacklisted journals, equal to an average of 0.5\u20130.9% of the total publications from Swedish HEIs. There was high variability between different HEI categories, with more established universities at the lower end of the spectrum, while university colleges and new universities had a much higher proportion (\u223c2%). There was a general decreasing trend during the study period (\u03c1 = 0.83) for all categories of HEIs. The study presents a methodology to identify questionable publishing in academia that could be applied to other countries with similar infrastructure. Thus, it could serve as a starting point for the development of a general framework for cross-national quantitative estimation of questionable publishing.<\/jats:p>","DOI":"10.1162\/qss_a_00033","type":"journal-article","created":{"date-parts":[[2020,3,25]],"date-time":"2020-03-25T13:06:31Z","timestamp":1585141591000},"page":"505-524","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":9,"title":["Methodological considerations for identifying questionable publishing in a national context: The case of Swedish Higher Education Institutions"],"prefix":"10.1162","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5196-7148","authenticated-orcid":false,"given":"Gustaf","family":"Nelhans","sequence":"first","affiliation":[{"name":"University of Bor\u00e5s, Sweden"},{"name":"University of Southern Denmark, Denmark"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2074-5857","authenticated-orcid":false,"given":"Theo","family":"Bodin","sequence":"additional","affiliation":[{"name":"Karolinska Institutet, Sweden"}]}],"member":"281","published-online":{"date-parts":[[2020,6,1]]},"reference":[{"key":"2025073014021606100_bib1","doi-asserted-by":"crossref","unstructured":"Asadi,  A., Rahbar,  N., Asadi,  M., Asadi,  F., & Paji,  K. K. (2017). Online-based approaches to identify real journals and publishers from hijacked ones. Science and Engineering Ethics, 23(1), 305\u2013308. https:\/\/doi.org\/10.1007\/s11948-015-9747-9","DOI":"10.1007\/s11948-015-9747-9"},{"key":"2025073014021606100_bib2","doi-asserted-by":"crossref","unstructured":"Bohannon,  J.\n           (2013). Who\u2019s afraid of peer review?Science, 342(6154), 60. https:\/\/doi.org\/10.1126\/science.342.6154.60","DOI":"10.1126\/science.342.6154.60"},{"key":"2025073014021606100_bib3","doi-asserted-by":"crossref","unstructured":"Bowker,  G. C., & Star,  S. L. (1999). Sorting things out: Classification and its consequences. Cambridge, MA: MIT Press.","DOI":"10.7551\/mitpress\/6352.001.0001"},{"key":"2025073014021606100_bib4","doi-asserted-by":"crossref","unstructured":"da Silva,  J. A. T.\n           (2017). Preprints: ethical hazard or academic liberation?Kome-an International Journal of Pure Communication Inquiry, 5(2), 73\u201380. https:\/\/doi.org\/10.17646\/kome.2017.26","DOI":"10.17646\/KOME.2017.26"},{"key":"2025073014021606100_bib5","doi-asserted-by":"crossref","unstructured":"Dadkhah,  M., & Maliszewski,  T. (2015). Hijacked journals\u2014threats and challenges to countries\u2019 scientific ranking. International Journal of Technology Enhanced Learning, 7(3), 281\u2013288. https:\/\/doi.org\/10.1504\/ijtel.2015.072819","DOI":"10.1504\/IJTEL.2015.072819"},{"key":"2025073014021606100_bib6","doi-asserted-by":"crossref","unstructured":"Das,  S., & Chatterjee,  S. S. (2018). Cabell\u2019s blacklist: A new way to tackle predatory journals. Indian Journal of Psychological Medicine, 40(2), 197\u2013198. https:\/\/doi.org\/10.4103\/IJPSYM.IJPSYM_290_17","DOI":"10.4103\/IJPSYM.IJPSYM_290_17"},{"key":"2025073014021606100_bib7","doi-asserted-by":"crossref","unstructured":"Demir,  S. B.\n           (2018a). Predatory journals: Who publishes in them and why?Journal of Informetrics, 12(4), 1296\u20131311. https:\/\/doi.org\/10.1016\/j.joi.2018.10.008","DOI":"10.1016\/j.joi.2018.10.008"},{"key":"2025073014021606100_bib8","doi-asserted-by":"crossref","unstructured":"Demir,  S. B.\n           (2018b). Pros and cons of the new financial support policy for Turkish researchers. Scientometrics, 116(3), 2053\u20132068. https:\/\/doi.org\/10.1007\/s11192-018-2833-4","DOI":"10.1007\/s11192-018-2833-4"},{"key":"2025073014021606100_bib9","doi-asserted-by":"crossref","unstructured":"Editorial. (2018). Nature, 562(308). https:\/\/doi.org\/10.1038\/d41586-018-07033-5","DOI":"10.1038\/d41586-018-07033-5"},{"key":"2025073014021606100_bib10","doi-asserted-by":"crossref","unstructured":"Erfanmanesh,  M., & Pourhossein,  R. (2017). Publishing in predatory open access journals: A case of Iran. Publishing Research Quarterly, 33(4), 433\u2013444. https:\/\/doi.org\/10.1007\/s12109-017-9547-y","DOI":"10.1007\/s12109-017-9547-y"},{"key":"2025073014021606100_bib11","doi-asserted-by":"crossref","unstructured":"Eriksson,  S., & Helgesson,  G. (2017). The false academy: Predatory publishing in science and bioethics. Medicine Health Care and Philosophy, 20(2), 163\u2013170. https:\/\/doi.org\/10.1007\/s11019-016-9740-3","DOI":"10.1007\/s11019-016-9740-3"},{"key":"2025073014021606100_bib12","doi-asserted-by":"crossref","unstructured":"Eykens,  J., Guns,  R., Rahman,  A. I. M. M., & Engels,  T. C. E. (2019). Identifying publications in questionable journals in the context of performance-based research funding. PLOS ONE, 14(11), e0224541. https:\/\/doi.org\/https:\/\/doi.org\/10.1371\/journal.pone.0224541","DOI":"10.1371\/journal.pone.0224541"},{"key":"2025073014021606100_bib13","unstructured":"Eykens,  J., Guns,  R., Rahman,  A. I. M. M., S\u012ble,  L., & Engels,  T. C. E. (2018). Predatory open access journals: A review of past screenings within the Flemish performance based research funding system (2014\u20132018). Proceedings of the 23rd International Conference on Science and Technology Indicators, Leiden."},{"key":"2025073014021606100_bib14","doi-asserted-by":"crossref","unstructured":"Ferris,  L. E., & Winker,  M. A. (2017). Ethical issues in publishing in predatory journals. Biochemia Medica, 27(2), 279\u2013284. https:\/\/doi.org\/10.11613\/bm.2017.030","DOI":"10.11613\/BM.2017.030"},{"key":"2025073014021606100_bib15","doi-asserted-by":"crossref","unstructured":"Frandsen,  T. F.\n           (2017). Are predatory journals undermining the credibility of science? A bibliometric analysis of citers. Scientometrics, 113(3), 1513\u20131528. https:\/\/doi.org\/10.1007\/s11192-017-2520-x","DOI":"10.1007\/s11192-017-2520-x"},{"key":"2025073014021606100_bib16","doi-asserted-by":"crossref","unstructured":"Gasparyan,  A. Y., Yessirkepov,  M., Voronov,  A. A., Gorin,  S. V., Koroleva,  A. M., & Kitas,  G. D. (2016). Statement on publication ethics for editors and publishers. Journal of Korean Medical Science, 31(9), 1351\u20131354. https:\/\/doi.org\/10.3346\/jkms.2016.31.9.1351","DOI":"10.3346\/jkms.2016.31.9.1351"},{"key":"2025073014021606100_bib17","doi-asserted-by":"crossref","unstructured":"Haider,  J., & \u00c5str\u00f6m,  F. (2017). Dimensions of trust in scholarly communication: Problematizing peer review in the aftermath of John Bohannon\u2019s \u201cSting\u201d in science. Journal of the Association for Information Science and Technology, 68(2), 450\u2013467. https:\/\/doi.org\/10.1002\/asi.23669","DOI":"10.1002\/asi.23669"},{"key":"2025073014021606100_bib18","unstructured":"Hansson,  G., Barriere,  S. G., Gurell,  J., Lindholm,  M., Lundin,  P., & Wikgren,  M. (2019). The Swedish research barometer 2019: The Swedish research system in international comparison (VR1915). Stockholm: Swedish Research Council."},{"key":"2025073014021606100_bib19","doi-asserted-by":"crossref","unstructured":"Hoffecker,  L.\n           (2018). Cabells Scholarly Analytics. Journal of the Medical Library Association, 106(2), 270\u2013272. https:\/\/doi.org\/10.5195\/jmla.2018.403","DOI":"10.5195\/jmla.2018.403"},{"key":"2025073014021606100_bib20","doi-asserted-by":"crossref","unstructured":"Jalalian,  M., & Dadkhah,  M. (2015). The full story of 90 hijacked journals from August 2011 to June 2015. Geographica Pannonica, 19(2), 73\u201387. https:\/\/doi.org\/10.5937\/GeoPan1502073J","DOI":"10.5937\/GeoPan1502073J"},{"key":"2025073014021606100_bib21","doi-asserted-by":"crossref","unstructured":"Josephson,  A., & Michler,  J. D. (2018). Viewpoint: Beasts of the field? Ethics in agricultural and applied economics. Food Policy, 79, 1\u201311. https:\/\/doi.org\/10.1016\/j.foodpol.2018.08.001","DOI":"10.1016\/j.foodpol.2018.08.001"},{"key":"2025073014021606100_bib22","doi-asserted-by":"crossref","unstructured":"Manca,  A., Moher,  D., Cugusi,  L., Dvir,  Z., & Deriu,  F. (2018). How predatory journals leak into PubMed. Canadian Medical Association Journal, 190(35), E1042\u2013E1045. https:\/\/doi.org\/10.1503\/cmaj.180154","DOI":"10.1503\/cmaj.180154"},{"key":"2025073014021606100_bib23","doi-asserted-by":"crossref","unstructured":"McLeod,  A., Savage,  A., & Simkin,  M. G. (2018). The ethics of predatory journals. Journal of Business Ethics, 153(1), 121\u2013131. https:\/\/doi.org\/10.1007\/s10551-016-3419-9","DOI":"10.1007\/s10551-016-3419-9"},{"key":"2025073014021606100_bib24","doi-asserted-by":"crossref","unstructured":"Mercier,  E., Tardif,  P. A., Moore,  L., Le Sage,  N., & Cameron,  P. A. (2018). Invitations received from potential predatory publishers and fraudulent conferences: A 12-month early-career researcher experience. Postgraduate Medical Journal, 94(1108), 104\u2013108. https:\/\/doi.org\/10.1136\/postgradmedj-2017-135097","DOI":"10.1136\/postgradmedj-2017-135097"},{"key":"2025073014021606100_bib25","doi-asserted-by":"crossref","unstructured":"Mukherjee,  B.\n           (2018). Measuring scientific value of Indian journals beyond impact factor: a case with physics-chemistry-biology journals. Current Science, 114(8), 1613\u20131623. https:\/\/doi.org\/10.18520\/cs\/v114\/i08\/1613-1623","DOI":"10.18520\/cs\/v114\/i08\/1613-1623"},{"key":"2025073014021606100_bib26","doi-asserted-by":"crossref","unstructured":"Oermann,  M. H., Conklin,  J. L., Nicoll,  L. H., Chinn,  P. L., Ashton,  K. S., Edie,  A. H., \u2026 Budinger,  S. C. (2016). Study of predatory open access nursing journals. Journal of Nursing Scholarship, 48(6), 624\u2013632.","DOI":"10.1111\/jnu.12248"},{"key":"2025073014021606100_bib27","doi-asserted-by":"crossref","unstructured":"Onder,  C., & Erdil,  S. E. (2017). Opportunities and opportunism: Publication outlet selection under pressure to increase research productivity. Research Evaluation, 26(2), 66\u201377. https:\/\/doi.org\/10.1093\/reseval\/rvx006","DOI":"10.1093\/reseval\/rvx006"},{"key":"2025073014021606100_bib28","doi-asserted-by":"crossref","unstructured":"Perlin,  M. S., Imasato,  T., & Borenstein,  D. (2018). Is predatory publishing a real threat? Evidence from a large database study. Scientometrics, 116(1), 255\u2013273.","DOI":"10.1007\/s11192-018-2750-6"},{"key":"2025073014021606100_bib29","doi-asserted-by":"crossref","unstructured":"Samal,  J., & Dehury,  R. K. (2017). The need and importance of incorporating academic research results into the curricula of Ayurveda in India. Journal of Clinical and Diagnostic Research, 11(6), KA01\u2013KA03. https:\/\/doi.org\/10.7860\/jcdr\/2017\/26742.10112","DOI":"10.7860\/JCDR\/2017\/26742.10112"},{"key":"2025073014021606100_bib30","doi-asserted-by":"crossref","unstructured":"Seethapathy,  G. S., Kumar,  J. U. S., & Hareesha,  A. S. (2016). India\u2019s scientific publication in predatory journals: need for regulating quality of Indian science and education. Current Science, 111(11), 1759\u20131764. https:\/\/doi.org\/10.18520\/cs\/v111\/i11\/1759-1764","DOI":"10.18520\/cs\/v111\/i11\/1759-1764"},{"key":"2025073014021606100_bib31","doi-asserted-by":"crossref","unstructured":"Shaghaei,  N., Wien,  C., Holck,  J., Thiesen,  A. L., Ellegaard,  O., Vlachos,  E., & Drachen,  T. (2018). Being a deliberate prey of a predator: Researchers\u2019 thoughts after having published in predatory journal. Liber Quarterly, 28(1).","DOI":"10.18352\/lq.10259"},{"key":"2025073014021606100_bib32","doi-asserted-by":"crossref","unstructured":"Shahri,  M. A., Jazi,  M. D., Borchardt,  G., & Dadkhah,  M. (2018). Detecting hijacked journals by using classification algorithms. Science and Engineering Ethics, 24(2), 655\u2013668. https:\/\/doi.org\/10.1007\/s11948-017-9914-2","DOI":"10.1007\/s11948-017-9914-2"},{"key":"2025073014021606100_bib33","doi-asserted-by":"crossref","unstructured":"Shen,  C., & Bj\u00f6rk,  B.-C. (2015). \u201cPredatory\u201d open access: a longitudinal study of article volumes and market characteristics. BMC Medicine, 13(1), 230.","DOI":"10.1186\/s12916-015-0469-2"},{"key":"2025073014021606100_bib34","doi-asserted-by":"crossref","unstructured":"S\u012ble,  L.\n           (2019). Entanglement of bibliographic database content and data collection practices: Rethinking data integration using findings from a European study. Procedia Computer Science, 146, 201\u2013207. https:\/\/doi.org\/10.1016\/j.procs.2019.01.094","DOI":"10.1016\/j.procs.2019.01.094"},{"key":"2025073014021606100_bib35","doi-asserted-by":"crossref","unstructured":"S\u012ble,  L., P\u00f6l\u00f6nen,  J., Sivertsen,  G., Guns,  R., Engels,  T. C. E., Arefiev,  P., \u2026 Teitelbaum,  R. (2018). Comprehensiveness of national bibliographic databases for social sciences and humanities: Findings from a European survey. Research Evaluation, 27(4), 310\u2013322. https:\/\/doi.org\/10.1093\/reseval\/rvy016","DOI":"10.1093\/reseval\/rvy016"},{"key":"2025073014021606100_bib36","doi-asserted-by":"crossref","unstructured":"Strielkowski,  W., Gryshova,  I., & Shcherbata,  M. (2017). Predatory publishing and Beall\u2019s list: lessons for the countries adapting novel research evaluation criteria. Science and Education, 8, 39\u201343. https:\/\/doi.org\/10.24195\/2414-4665-2017-8-5","DOI":"10.24195\/2414-4665-2017-8-5"},{"key":"2025073014021606100_bib37","doi-asserted-by":"crossref","unstructured":"Strinzel,  M., Severin,  A., Milzow,  K., & Egger,  M. (2019). \u201cBlacklists\u201d and \u201cwhitelists\u201d to tackle predatory publishing: A cross-sectional comparison and thematic analysis. PeerJ Preprints.","DOI":"10.7287\/peerj.preprints.27532v1"},{"key":"2025073014021606100_bib38","unstructured":"Testa,  J.\n           (n.d.). Journal selection process. Retrieved from https:\/\/clarivate.com\/webofsciencegroup\/essays\/journal-selection-process\/"},{"key":"2025073014021606100_bib39","doi-asserted-by":"crossref","unstructured":"van Eck,  N., & Waltman,  L. (2009). Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics, 84(2), 523\u2013538.","DOI":"10.1007\/s11192-009-0146-3"},{"key":"2025073014021606100_bib40","unstructured":"Vershinina,  S., Tarasova,  O., & Strielkowski,  W. (2017). Academic publishing, journal rankings, and scientific productivity. Terra Economicus, 15(4), 127\u2013135. https:\/\/doi.org\/10.23683\/2073-6606-2017-15-4-127-135"},{"key":"2025073014021606100_bib41","doi-asserted-by":"crossref","unstructured":"Yessirkepov,  M., Nurmashev,  B., & Anartayeva,  M. (2015). A Scopus-based analysis of publication activity in Kazakhstan from 2010 to 2015: Positive trends, concerns, and possible solutions. Journal of Korean Medical Science, 30(12), 1915\u20131919. https:\/\/doi.org\/10.3346\/jkms.2015.30.12.1915","DOI":"10.3346\/jkms.2015.30.12.1915"}],"container-title":["Quantitative Science Studies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/qss\/article-pdf\/1\/2\/505\/1885789\/qss_a_00033.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/qss\/article-pdf\/1\/2\/505\/1885789\/qss_a_00033.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T18:02:38Z","timestamp":1753898558000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/qss\/article\/1\/2\/505\/96137\/Methodological-considerations-for-identifying"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020]]},"references-count":41,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2020,6,1]]}},"URL":"https:\/\/doi.org\/10.1162\/qss_a_00033","relation":{},"ISSN":["2641-3337"],"issn-type":[{"type":"electronic","value":"2641-3337"}],"subject":[],"published-other":{"date-parts":[[2020]]},"published":{"date-parts":[[2020]]}}}