{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,12]],"date-time":"2025-12-12T10:05:32Z","timestamp":1765533932293,"version":"3.48.0"},"reference-count":43,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2025,12,12]],"date-time":"2025-12-12T00:00:00Z","timestamp":1765497600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012190","name":"Ministry of Science and Higher Education of the Russian Federation","doi-asserted-by":"publisher","award":["075-15-2024-544"],"award-info":[{"award-number":["075-15-2024-544"]}],"id":[{"id":"10.13039\/501100012190","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BDCC"],"abstract":"<jats:p>The reliable and early detection of promising research directions is of great practical importance, especially in cases of limited resources. It enables researchers, funding experts, and science authorities to focus their efforts effectively. Although citation analysis has been commonly considered the primary tool to detect directions for a long time, it lacks responsiveness, as it requires time for citations to emerge. In this paper, we propose a conceptual framework that detects new research directions with a contextual Top2Vec model, collects and analyzes reviews for those directions via Transformer-based classifiers, ranks them, and generates short summaries for the highest-scoring ones with a BART model. Averaging review scores for a whole topic helps mitigate the review bias problem. Experiments on past ICLR open reviews show that the highly ranked directions detected are significantly better cited; additionally, in most cases, they exhibit better publication dynamics.<\/jats:p>","DOI":"10.3390\/bdcc9120319","type":"journal-article","created":{"date-parts":[[2025,12,12]],"date-time":"2025-12-12T09:37:48Z","timestamp":1765532268000},"page":"319","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Identifying New Promising Research Directions with Open Peer Reviews and Contextual Top2Vec"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0811-725X","authenticated-orcid":false,"given":"Dmitry","family":"Devyatkin","sequence":"first","affiliation":[{"name":"Federal Research Center \u201cComputer Science and Control\u201d, Russian Academy of Sciences, Moscow 119333, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3113-3765","authenticated-orcid":false,"given":"Ilya V.","family":"Sochenkov","sequence":"additional","affiliation":[{"name":"Federal Research Center \u201cComputer Science and Control\u201d, Russian Academy of Sciences, Moscow 119333, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-5010-6487","authenticated-orcid":false,"given":"Dmitrii","family":"Popov","sequence":"additional","affiliation":[{"name":"Federal Research Center \u201cComputer Science and Control\u201d, Russian Academy of Sciences, Moscow 119333, Russia"}]},{"given":"Denis","family":"Zubarev","sequence":"additional","affiliation":[{"name":"Federal Research Center \u201cComputer Science and Control\u201d, Russian Academy of Sciences, Moscow 119333, Russia"}]},{"given":"Anastasia","family":"Ryzhova","sequence":"additional","affiliation":[{"name":"Federal Research Center \u201cComputer Science and Control\u201d, Russian Academy of Sciences, Moscow 119333, Russia"}]},{"given":"Fyodor","family":"Abanin","sequence":"additional","affiliation":[{"name":"Institute of Information Technologies, MIREA\u2014Russian Technological University, Moscow 119454, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9660-2396","authenticated-orcid":false,"given":"Oleg","family":"Grigoriev","sequence":"additional","affiliation":[{"name":"Federal Research Center \u201cComputer Science and Control\u201d, Russian Academy of Sciences, Moscow 119333, Russia"}]}],"member":"1968","published-online":{"date-parts":[[2025,12,12]]},"reference":[{"key":"ref_1","unstructured":"Chee, K.N., and Sanmugam, M. (2023). A Bibliometric Review of Studies on the Application of Augmented Reality to Cultural Heritage by Using Biblioshiny and CiteSpace. Embracing Cutting-Edge Technology in Modern Educational Settings, IGI Global."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Park, I., and Yoon, B. (2018). Identifying Promising Research Frontiers of Pattern Recognition through Bibliometric Analysis. Sustainability, 10.","DOI":"10.3390\/su10114055"},{"key":"ref_3","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., and Polosukhin, I. (2017, January 7\u20139). Attention is all you need. Proceedings of the 30th Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"683","DOI":"10.1007\/s11280-022-01109-z","article-title":"What have we learned from OpenReview?","volume":"26","author":"Wang","year":"2023","journal-title":"World Wide Web"},{"key":"ref_5","unstructured":"Kumar, P.B., Ranjan, S., Ghosal, T., Agrawal, M., and Ekbal, A. (2021, January 1\u20133). PEERAssist: Leveraging on paper-review interactions to predict peer review decisions. Proceedings of the 23rd International Conference Towards Open and Trustworthy Digital Societies, Virtual Event."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Angelov, D., and Inkpen, D. (2024, January 12\u201316). Topic modeling: Contextual token embeddings are all you need. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP, Miami, FL, USA.","DOI":"10.18653\/v1\/2024.findings-emnlp.790"},{"key":"ref_7","unstructured":"(2025, October 21). ArXiv.org. Open-Access Archive. Available online: https:\/\/arxiv.org."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1108\/IJCS-01-2021-0001","article-title":"Identification of data mining research frontier based on conference papers","volume":"5","author":"Huang","year":"2021","journal-title":"Int. J. Crowd Sci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"101421","DOI":"10.1016\/j.joi.2023.101421","article-title":"Research frontier detection and analysis based on research grants information: A case study on health informatics in the US","volume":"17","author":"Ye","year":"2023","journal-title":"J. Informetr."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Albrekht, V., Mukhamediev, R.I., Popova, Y., Muhamedijeva, E., and Botaibekov, A. (2025). Top2Vec Topic Modeling to Analyze the Dynamics of Publication Activity Related to Environmental Monitoring Using Unmanned Aerial Vehicles. Publications, 13.","DOI":"10.3390\/publications13020015"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"83052","DOI":"10.1109\/ACCESS.2023.3290906","article-title":"Predicting the future popularity of academic publications using deep learning by considering it as temporal citation networks","volume":"11","author":"Abbas","year":"2023","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"e2118046119","DOI":"10.1073\/pnas.2118046119","article-title":"Is novel research worth doing? Evidence from peer review at 49 journals","volume":"119","author":"Teplitskiy","year":"2022","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Severin, A., Strinzel, M., Egger, M., Barros, T., Sokolov, A., Mouatt, J.V., and M\u00fcller, S. (2023). Relationship between journal impact factor and the thoroughness and helpfulness of peer reviews. PLoS Biol., 21.","DOI":"10.1371\/journal.pbio.3002238"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1126\/science.aaa0185","article-title":"Big names or big ideas: Do peer-review panels select the best science proposals?","volume":"348","author":"Danielle","year":"2015","journal-title":"Science"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yang, C., and Han, J. (2023, January 3\u20137). Revisiting citation prediction with cluster-aware text-enhanced heterogeneous graph neural networks. Proceedings of the IEEE 39th International Conference on Data Engineering (ICDE 2023), Anaheim, CA, USA.","DOI":"10.1109\/ICDE55515.2023.00058"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Staudinger, M., Kusa, W., Piroi, F., and Hanbury, A. (2024, January 16). An analysis of tasks and datasets in peer reviewing. Proceedings of the Fourth Workshop on Scholarly Document Processing (SDP 2024), Bangkok, Thailand.","DOI":"10.18653\/v1\/2024.sdp-1.24"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1007\/s00799-023-00382-1","article-title":"Enhancing the examination of obstacles in an automated peer review system","volume":"25","author":"Fernandes","year":"2023","journal-title":"Int. J. Digit. Libr."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kang, D., Ammar, W., Dalvi, B., Van Zuylen, M., Kohlmeier, S., Hovy, E., and Schwartz, R. (2018, January 1\u20136). A dataset of peer reviews (PeerRead): Collection, insights and NLP applications. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), New Orleans, LA, USA.","DOI":"10.18653\/v1\/N18-1149"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ribeiro, A.C., Sizo, A., Cardoso, H.L., and Reis, L.P. (2021, January 7\u20139). Acceptance decision prediction in peer review through sentiment analysis. Proceedings of the 20th EPIA Conference on Artificial Intelligence, Virtual Event.","DOI":"10.1007\/978-3-030-86230-5_60"},{"key":"ref_20","unstructured":"Plank, B., and Van Dalen, R. (2019, January 25). CiteTracked: A Longitudinal Dataset of Peer Reviews and Citations. Proceedings of the Joint Workshop on Bibliometric-Enhanced Information Retrieval and Natural Language Processing for Digital Libraries (ACM BIRNDL SIGIR), Paris, France."},{"key":"ref_21","unstructured":"Szumega, J., Bougueroua, L., Gkotse, B., Jouvelot, P., and Ravotti, F. (2023). The open review-based (ORB) dataset: Towards automatic assessment of scientific papers and experiment proposals in high-energy physics. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1162\/tacl_a_00325","article-title":"Topic modeling in embedding spaces","volume":"8","author":"Dieng","year":"2020","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_23","unstructured":"Angelov, D. (2022). Top2vec: Distributed representations of topics. arXiv."},{"key":"ref_24","unstructured":"Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv."},{"key":"ref_25","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2\u20137). Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA."},{"key":"ref_26","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"J. Mach. Learn. Res."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., and Zettlemoyer, L. (2020, January 5\u201310). BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"ref_28","unstructured":"Ranganathan, J., and Abuka, G. (December, January 29). Text Summarization using Transformer Model. Proceedings of the Ninth International Conference on Social Networks Analysis, Management and Security (SNAMS), Milan, Italy."},{"key":"ref_29","unstructured":"Premnath, P., Yenumulapalli, V.O., Mohankumar, P., and Sivanaiah, R. (August, January 31). TechSSN at SemEval-2025 Task 10: A Comparative Analysis of Transformer Models for Dominant Narrative-Based News Summarization. Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), Vienna, Austria."},{"key":"ref_30","unstructured":"Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., and Artzi, Y. (May, January 26). BERTScore: Evaluating Text Generation with BERT. Proceedings of the International Conference on Learning Representations (ICLR), Virtual Event."},{"key":"ref_31","unstructured":"Gilhuly, C., and Shahzad, H. (2025). Consistency Evaluation of News Article Summaries Generated by Large (and Small) Language Models. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"102080","DOI":"10.1016\/j.jksuci.2024.102080","article-title":"FuzzyTP-BERT: Enhancing extractive text summarization with fuzzy topic modeling and transformer networks","volume":"36","author":"Onan","year":"2024","journal-title":"J. King Saud Univ. Comput. Inf. Sci."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Wang, Z., Duan, Z., Zhang, H., Wang, C., Tian, L., Chen, B., and Zhou, M. (2020, January 16\u201320). Friendly topic assistant for transformer based abstractive summarization. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.","DOI":"10.18653\/v1\/2020.emnlp-main.35"},{"key":"ref_34","unstructured":"(2025, October 10). ICORE Conference Portal. Available online: https:\/\/portal.core.edu.au\/conf-ranks."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzm\u00e1n, F., and Stoyanov, V. (2020, January 5\u201310). Unsupervised Cross-lingual Representation Learning at Scale. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2020.acl-main.747"},{"key":"ref_36","unstructured":"(2025, October 21). COntext TOp2Vec Frontiers. Available online: https:\/\/github.com\/masterdoors\/COTOF\/tree\/main."},{"key":"ref_37","first-page":"31","article-title":"Normalized (pointwise) mutual information in collocation extraction","volume":"Volume 30","author":"Bouma","year":"2009","journal-title":"Proceedings of the Biennial GSCL Conference"},{"key":"ref_38","unstructured":"(2025, October 10). ISA-NLP Linguistic Parser. Available online: https:\/\/github.com\/IINemo\/isanlp."},{"key":"ref_39","unstructured":"(2025, October 21). Novel or Not Dataset. Available online: https:\/\/github.com\/masterdoors\/COTOF\/blob\/main\/notebooks\/new_or_not_eng.ipynb."},{"key":"ref_40","unstructured":"(2025, October 21). Summarization Dataset. Available online: https:\/\/github.com\/masterdoors\/COTOF\/tree\/main\/datasets\/summarization."},{"key":"ref_41","unstructured":"(2025, October 21). OCTIS. Available online: https:\/\/github.com\/MIND-Lab\/OCTIS."},{"key":"ref_42","unstructured":"(2025, October 21). Review Segmentation Model. Available online: https:\/\/huggingface.co\/Ryzhik22\/rev-classif-twolangs."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/j.yrtph.2019.01.003","article-title":"Science peer review for the 21st century: Assessing scientific consensus for decision-making while managing conflict of interests, reviewer and process bias","volume":"103","author":"Kirman","year":"2019","journal-title":"Regul. Toxicol. Pharmacol."}],"container-title":["Big Data and Cognitive Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/12\/319\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,12]],"date-time":"2025-12-12T10:01:10Z","timestamp":1765533670000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/12\/319"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,12]]},"references-count":43,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["bdcc9120319"],"URL":"https:\/\/doi.org\/10.3390\/bdcc9120319","relation":{},"ISSN":["2504-2289"],"issn-type":[{"value":"2504-2289","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,12]]}}}