{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,13]],"date-time":"2026-07-13T14:50:56Z","timestamp":1783954256911,"version":"3.55.0"},"reference-count":47,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T00:00:00Z","timestamp":1766102400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Education","award":["23YJC870009"],"award-info":[{"award-number":["23YJC870009"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Publications"],"abstract":"<jats:p>In response to the persistent imbalance between the rapid expansion of scholarly publishing and the constrained availability of qualified reviewers, an empirical investigation was conducted to examine the feasibility and boundary conditions of employing Large Language Models (LLMs) in journal peer review. A parallel corpus of 493 pairs of human expert reviews and GPT-4o-generated reviews was constructed from the open peer-review platform PeerJ Computer Science. Analytical techniques, including keyword co-occurrence analysis, sentiment and subjectivity assessment, syntactic complexity measurement, and n-gram distributional entropy analysis, were applied to compare cognitive patterns, evaluative tendencies, and thematic coverage between human and AI reviewers. The results indicate that human and AI reviews exhibit complementary functional orientations. Human reviewers were observed to provide integrative and socially contextualized evaluations, while AI reviews emphasized structural verification and internal consistency, especially regarding the correspondence between abstracts and main texts. Contrary to the assumption of excessive leniency, GPT-4o-generated reviews demonstrated higher critical density and functional rigor, maintaining substantial topical alignment with human feedback. Based on these findings, a collaborative human\u2013AI review framework is proposed, in which AI systems are positioned as analytical assistants that conduct structured verification prior to expert evaluation. Such integration is expected to enhance the efficiency, consistency, and transparency of the peer-review process and to promote the sustainable development of scholarly communication.<\/jats:p>","DOI":"10.3390\/publications14010001","type":"journal-article","created":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T11:03:46Z","timestamp":1766142226000},"page":"1","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Human\u2013AI Complementarity in Peer Review: Empirical Analysis of PeerJ Data and Design of an Efficient Collaborative Review Framework"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4227-5590","authenticated-orcid":false,"given":"Zhihe","family":"Yang","sequence":"first","affiliation":[{"name":"College of Information Management, Heilongjiang University, Harbin 150080, China"},{"name":"Research Center for Information Resources Management, Heilongjiang University, Harbin 150080, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-2826-1304","authenticated-orcid":false,"given":"Xiaoyu","family":"Zhou","sequence":"additional","affiliation":[{"name":"College of Information Management, Heilongjiang University, Harbin 150080, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuxin","family":"Jiang","sequence":"additional","affiliation":[{"name":"College of Information Management, Heilongjiang University, Harbin 150080, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xinjie","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Information Management, Heilongjiang University, Harbin 150080, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qihui","family":"Gao","sequence":"additional","affiliation":[{"name":"College of Information Management, Heilongjiang University, Harbin 150080, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanzhu","family":"Lu","sequence":"additional","affiliation":[{"name":"College of Information Management, Heilongjiang University, Harbin 150080, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anqi","family":"Yang","sequence":"additional","affiliation":[{"name":"College of Information Management, Heilongjiang University, Harbin 150080, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,12,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Brown, P., and Levinson, S. C. (1987). Politeness: Some universals in language usage, Cambridge University Press.","DOI":"10.1017\/CBO9780511813085"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/03610927408827101","article-title":"A dendrite method for cluster analysis","volume":"3","author":"Harabasz","year":"1974","journal-title":"Communications in Statistics-Theory and Methods"},{"key":"ref_3","unstructured":"Cell Press (2025, October 19). Information for reviewers, Available online: https:\/\/www.cell.com\/reviewers."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1057\/s41599-020-00703-8","article-title":"AI-assisted peer review","volume":"8","author":"Checco","year":"2021","journal-title":"Humanities and Social Sciences Communications"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1109\/TPAMI.1979.4766909","article-title":"A cluster separation measure","volume":"PAMI-1","author":"Davies","year":"1979","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"92","DOI":"10.5334\/irsp.82","article-title":"Why psychologists should by default use Welch\u2019s t-test instead of Student\u2019s t-test","volume":"30","author":"Delacre","year":"2017","journal-title":"International Review of Social Psychology"},{"key":"ref_7","unstructured":"Dove Medical Press (2025, October 19). Peer review guidelines, Available online: https:\/\/www.dovepress.com\/peer-review-guidelines."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1383","DOI":"10.1007\/s10805-025-09604-4","article-title":"Exploring the impact of generative AI on peer review: Insights from journal reviewers","volume":"23","author":"Ebadi","year":"2025","journal-title":"Journal of Academic Ethics"},{"key":"ref_9","unstructured":"Elsevier (2025, October 17). The use of generative AI and AI-assisted technologies in the review process for Elsevier, Available online: https:\/\/www.elsevier.com\/about\/policies-and-standards\/the-use-of-generative-ai-and-ai-assisted-technologies-in-the-review-process."},{"key":"ref_10","unstructured":"Explosion AI (2025, October 18). Industrial-strength natural language processing in Python, Available online: https:\/\/spacy.io."},{"key":"ref_11","unstructured":"Frontiers (2025, October 17). Editorial policies and publication ethics, Available online: https:\/\/www.frontiersin.org\/guidelines\/policies-and-publication-ethics."},{"key":"ref_12","first-page":"92","article-title":"Sulla determinazione empirica delle leggi di probabilit\u00e0","volume":"4","author":"Glivenko","year":"1933","journal-title":"Giornale dell\u2019Istituto Italiano degli Attuari"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1136\/amiajnl-2011-000089","article-title":"Automation bias: A systematic review of frequency, effect mediators, and mitigators","volume":"19","author":"Goddard","year":"2012","journal-title":"Journal of the American Medical Informatics Association"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"100968","DOI":"10.1016\/j.patter.2024.100968","article-title":"The landscape of biomedical research","volume":"5","author":"Schmidt","year":"2024","journal-title":"Patterns"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"884","DOI":"10.1109\/TSE.2024.3366753","article-title":"Factoring expertise, workload, and turnover into code review recommendation","volume":"50","author":"Hajari","year":"2024","journal-title":"IEEE Transactions on Software Engineering"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1186\/s41073-023-00133-5","article-title":"Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review","volume":"8","author":"Hosseini","year":"2023","journal-title":"Research Integrity and Peer Review"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1609\/icwsm.v8i1.14550","article-title":"VADER: A parsimonious rule-based model for sentiment analysis of social media text","volume":"8","author":"Hutto","year":"2014","journal-title":"Proceedings of the International AAAI Conference on Web and Social Media"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/S1060-3743(01)00038-8","article-title":"Sugaring the pill: Praise and criticism in written feedback","volume":"10","author":"Hyland","year":"2001","journal-title":"Journal of Second Language Writing"},{"key":"ref_19","unstructured":"ICLR (2025, October 16). Leveraging LLM feedback to enhance review quality, Available online: https:\/\/blog.iclr.cc\/2025\/04\/15\/leveraging-llm-feedback-to-enhance-review-quality."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Jawahar, G., Abdul-Mageed, M., and Lakshmanan, L. V. S. (2020). Automatic detection of machine generated text: A critical survey. arXiv.","DOI":"10.18653\/v1\/2020.coling-main.208"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1016\/0020-0190(89)90102-6","article-title":"An algorithm for drawing general undirected graphs","volume":"31","author":"Kamada","year":"1989","journal-title":"Information Processing Letters"},{"key":"ref_22","unstructured":"Kuznetsov, I., Afzal, O. M., Dercksen, K., and Gurevych, I. (2024). What can natural language processing do for peer review. arXiv."},{"key":"ref_23","unstructured":"Latona, G. R., Ribeiro, M. H., Davidson, T. R., and West, R. (2024). The AI review lottery: Widespread AI-assisted peer reviews boost paper scores and acceptance rates. arXiv."},{"key":"ref_24","unstructured":"Liang, W., Izzo, Z., Zhang, Y., Leppert, H., and Zou, J. (2024a). Monitoring AI-modified content at scale: A case study on the impact of ChatGPT on AI conference peer reviews. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"AIoa2400196","DOI":"10.1056\/AIoa2400196","article-title":"Can large language models provide useful feedback on research papers? A large-scale empirical analysis","volume":"1","author":"Liang","year":"2024b","journal-title":"NEJM AI"},{"key":"ref_26","unstructured":"Loria, S. (2025, October 15). TextBlob: Simplified text processing, Available online: https:\/\/textblob.readthedocs.io\/en\/dev\/."},{"key":"ref_27","unstructured":"Mann, S. P., Aboy, M., Seah, J. J., and Cohen, I. G. (2025). AI and the future of academic peer review. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"McInnes, L., Healy, J., and Melville, J. (2018). UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv.","DOI":"10.21105\/joss.00861"},{"key":"ref_29","unstructured":"MDPI (2025, October 15). Annual report 2023, Available online: https:\/\/www.mdpi.com\/annual-report-2023."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1177\/17470161231224552","article-title":"Death of a reviewer or death of peer review integrity? The challenges of using AI tools in peer reviewing and the need to go beyond publishing policies","volume":"20","author":"Mollaki","year":"2024","journal-title":"Research Ethics"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"852","DOI":"10.1038\/d41586-025-00894-7","article-title":"AI is transforming peer review: And many scientists are worried","volume":"639","author":"Naddaf","year":"2025","journal-title":"Nature"},{"key":"ref_32","unstructured":"National Library of Medicine (2025, October 18). MEDLINE\u00ae citation counts by year of publication, Available online: https:\/\/www.nlm.nih.gov\/bsd\/medline_cit_counts_yr_pub.html."},{"key":"ref_33","unstructured":"Ouvrir la Science (2025, October 17). Excessive growth in the number of scientific publications, Available online: https:\/\/www.ouvrirlascience.fr\/excessive-growth-in-the-number-of-scientific-publications."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1038\/s41586-022-05543-x","article-title":"Papers and patents are becoming less disruptive over time","volume":"613","author":"Park","year":"2023","journal-title":"Nature"},{"key":"ref_35","unstructured":"PeerJ (2025, October 16). Peer-review timeline, Available online: https:\/\/peerj.com\/benefits\/peer-review-timeline."},{"key":"ref_36","unstructured":"PeerJ (2025, December 02). Policies & procedures, Available online: https:\/\/peerj.com\/about\/policies-and-procedures\/."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Reimers, N., and Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using siamese BERT-networks. arXiv.","DOI":"10.18653\/v1\/D19-1410"},{"key":"ref_38","unstructured":"Richardson, L. (2025, October 15). Beautiful soup documentation, Available online: https:\/\/www.crummy.com\/software\/BeautifulSoup\/bs4\/doc."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/0377-0427(87)90125-7","article-title":"Silhouettes: A graphical aid to the interpretation and validation of cluster analysis","volume":"20","author":"Rousseeuw","year":"1987","journal-title":"Journal of Computational and Applied Mathematics"},{"key":"ref_40","unstructured":"Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1002\/j.1538-7305.1951.tb01366.x","article-title":"Prediction and entropy of printed English","volume":"30","author":"Shannon","year":"1951","journal-title":"Bell System Technical Journal"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Shopovski, J., Mohdali, R., and Marolov, D. (2025). Revolutionizing Peer Review: A Comparative Analysis of ChatGPT and Human Review Reports in Scientific Publishing. Preprints, 2025020058.","DOI":"10.20944\/preprints202502.0058.v1"},{"key":"ref_43","unstructured":"Springer Nature (2025, October 14). Peer review policy, process and guidance, Available online: https:\/\/www.springer.com\/gp\/editorial-policies\/peer-review-policy-process."},{"key":"ref_44","unstructured":"STM (2025, October 16). OA Dashboard: Open access data and the publishing industry, Available online: https:\/\/www.stm-assoc.org\/oa-dashboard\/oa-dashboard-2024."},{"key":"ref_45","unstructured":"Wiley (2025, October 14). Best practice guidelines on research integrity and publishing ethics, Available online: https:\/\/authorservices.wiley.com\/ethics-guidelines\/index.html."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"e2926","DOI":"10.7717\/peerj-cs.2926","article-title":"RiceChain-Plus: An enhanced framework for blockchain-based rice supply chain systems-ensuring security, privacy, and efficiency","volume":"11","author":"Yakubu","year":"2025","journal-title":"PeerJ Computer Science"},{"key":"ref_47","unstructured":"Zheng, X., and Xiong, T. (, January March). Evaluating the language style of AI-generated peer review. iConference 2025 Proceedings, Virtual."}],"container-title":["Publications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2304-6775\/14\/1\/1\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,21]],"date-time":"2025-12-21T05:16:20Z","timestamp":1766294180000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2304-6775\/14\/1\/1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,19]]},"references-count":47,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["publications14010001"],"URL":"https:\/\/doi.org\/10.3390\/publications14010001","relation":{},"ISSN":["2304-6775"],"issn-type":[{"value":"2304-6775","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,19]]}}}