{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T05:10:42Z","timestamp":1780981842985,"version":"3.54.1"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2021,5,23]],"date-time":"2021-05-23T00:00:00Z","timestamp":1621728000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,5,23]],"date-time":"2021-05-23T00:00:00Z","timestamp":1621728000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100019018","name":"CAMS Initiative for Innovative Medicine","doi-asserted-by":"crossref","award":["2016-I2M-3-018"],"award-info":[{"award-number":["2016-I2M-3-018"]}],"id":[{"id":"10.13039\/501100019018","id-type":"DOI","asserted-by":"crossref"}]},{"name":"NSTL","award":["2020XM05"],"award-info":[{"award-number":["2020XM05"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Scientometrics"],"published-print":{"date-parts":[[2021,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Breakthrough research in scientific fields usually comes as a manifestation of major development and advancement. These advances build to an epiphany where new ways of thinking about a problem become possible. Identifying breakthrough research can be useful for cultivating and funding further innovation. This article presents a new method for identifying scientific breakthroughs from research papers based on cue words commonly associated with major advancements. We looked for specific terms signifying scientific breakthroughs in citing sentences to identify breakthrough articles. By setting a threshold for the number of citing sentences (\u201ccitances\u201d) with breakthrough cue words that peer scholars often use when evaluating research, we identified articles containing breakthrough research. We call this approach the \u201cothers-evaluation\u201d process. We then shortlisted candidates from the selected articles based on the authors\u2019 evaluations of their own research, found in the abstracts. This we call the \u201cself-evaluation\u201d process. Combining the two approaches into a dual \u201cothers-self\u201d evaluation process, we arrived at a sample of 237 potential breakthrough articles, most of which are recommended by the Faculty Opinions. Based on the breakthrough articles identified, using SVM, TextCNN, and BERT to train the models to identify abstracts with breakthrough evaluations. This automatic identification model can greatly simplify the process of others-self-evaluation process and promote identifying breakthrough research.<\/jats:p>","DOI":"10.1007\/s11192-021-04003-z","type":"journal-article","created":{"date-parts":[[2021,5,23]],"date-time":"2021-05-23T09:02:35Z","timestamp":1621760555000},"page":"5531-5549","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["A deep learning approach for identifying biomedical breakthrough discoveries using context analysis"],"prefix":"10.1007","volume":"126","author":[{"given":"Xue","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuemei","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jian","family":"Du","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuwen","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiao","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6946-3482","authenticated-orcid":false,"given":"Xiaoli","family":"Tang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,5,23]]},"reference":[{"key":"4003_CR1","unstructured":"Abu-Jbara, A., Ezra, J., & Radev, D. (2013). Purpose and polarity of citation: Towards NLP-based bibliometrics. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, Georgia, June 2013 (pp. 596\u2013606): Association for Computational Linguistics."},{"issue":"3","key":"4003_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/1961189.1961199","volume":"2","author":"C-C Chang","year":"2011","unstructured":"Chang, C.-C., & Lin, C.-J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3), 1\u201327. https:\/\/doi.org\/10.1145\/1961189.1961199","journal-title":"ACM Transactions on Intelligent Systems and Technology"},{"issue":"3","key":"4003_CR3","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1002\/asi.21694","volume":"63","author":"C Chen","year":"2012","unstructured":"Chen, C. (2012). Predictive effects of structural variation on citation counts. Journal of the American Society for Information Science and Technology, 63(3), 431\u2013449. https:\/\/doi.org\/10.1002\/asi.21694","journal-title":"Journal of the American Society for Information Science and Technology"},{"issue":"3","key":"4003_CR4","doi-asserted-by":"publisher","first-page":"191","DOI":"10.1016\/j.joi.2009.03.004","volume":"3","author":"C Chen","year":"2009","unstructured":"Chen, C., Chen, Y., Horowitz, M., Hou, H., Liu, Z., & Pellegrino, D. (2009). Towards an explanatory and computational theory of scientific discovery. Journal of Informetrics, 3(3), 191\u2013209. https:\/\/doi.org\/10.1016\/j.joi.2009.03.004","journal-title":"Journal of Informetrics"},{"key":"4003_CR5","doi-asserted-by":"publisher","first-page":"246","DOI":"10.1016\/j.future.2013.07.018","volume":"32","author":"JQ Chen","year":"2014","unstructured":"Chen, J. Q., & Zhuge, H. (2014). Summarization of scientific documents by detecting common facts in citations. Future Generation Computer Systems-the International Journal of Escience, 32, 246\u2013252. https:\/\/doi.org\/10.1016\/j.future.2013.07.018","journal-title":"Future Generation Computer Systems-the International Journal of Escience"},{"issue":"6","key":"4003_CR7","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1108\/eb050097","volume":"19","author":"C Cleverdon","year":"1967","unstructured":"Cleverdon, C. (1967). The Cranfield tests on index language devices. Aslib Proceedings, 19(6), 173\u2013194. https:\/\/doi.org\/10.1108\/eb050097","journal-title":"Aslib Proceedings"},{"issue":"2","key":"4003_CR8","doi-asserted-by":"publisher","first-page":"575","DOI":"10.1007\/s11192-015-1598-2","volume":"104","author":"JA Comins","year":"2015","unstructured":"Comins, J. A., & Hussey, T. W. (2015). Detecting seminal research contributions to the development and use of the global positioning system by reference publication year spectroscopy. Scientometrics, 104(2), 575\u2013580. https:\/\/doi.org\/10.1007\/s11192-015-1598-2","journal-title":"Scientometrics"},{"issue":"3","key":"4003_CR9","doi-asserted-by":"publisher","first-page":"1495","DOI":"10.1007\/s11192-016-2238-1","volume":"110","author":"JA Comins","year":"2017","unstructured":"Comins, J. A., & Leydesdorff, L. (2017). Citation algorithms for identifying research milestones driving biomedical innovation. Scientometrics, 110(3), 1495\u20131504. https:\/\/doi.org\/10.1007\/s11192-016-2238-1","journal-title":"Scientometrics"},{"key":"4003_CR10","unstructured":"Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv e-prints, http:\/\/arxiv.org\/abs\/1810.04805"},{"issue":"1","key":"4003_CR11","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1002\/asi.20707","volume":"59","author":"A Elkiss","year":"2008","unstructured":"Elkiss, A., Shen, S., Fader, A., Erkan, G., States, D., & Radev, D. (2008). Blind men and elephants: What do citation summaries tell us about a research article? Journal of the American Society for Information Science and Technology, 59(1), 51\u201362. https:\/\/doi.org\/10.1002\/asi.20707","journal-title":"Journal of the American Society for Information Science and Technology"},{"key":"4003_CR12","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1186\/s13326-015-0037-x","volume":"6","author":"T Fujiwara","year":"2015","unstructured":"Fujiwara, T., & Yamamoto, Y. (2015). CoLiL: a database and search service for citation contexts in the life sciences domain. Journal of Biomedical Semantics, 6, 38. https:\/\/doi.org\/10.1186\/s13326-015-0037-x","journal-title":"Journal of Biomedical Semantics"},{"issue":"2","key":"4003_CR13","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1007\/bf02163625","volume":"13","author":"E Garfield","year":"1992","unstructured":"Garfield, E., & Welljams-Dorof, A. (1992). Of Nobel class: A citation perspective on high impact research authors. Theoretical Medicine, 13(2), 117\u2013135. https:\/\/doi.org\/10.1007\/bf02163625","journal-title":"Theoretical Medicine"},{"key":"4003_CR14","unstructured":"Guo, Q., Du, J., & Tang, X. A Bibliometric Framework for Identifying \u201cAcademic Chain\u201d \u2014A Case Study of 2014 Nobel Prize for Chemistry. In Proceedings of the 23rd international conference on science and technology indicators, 2018-09-11 (pp. 331\u2013338): Centre for Science and Technology Studies (CWTS)."},{"issue":"3","key":"4003_CR15","doi-asserted-by":"publisher","first-page":"1645","DOI":"10.1007\/s11192-018-2944-y","volume":"117","author":"S-U Hassan","year":"2018","unstructured":"Hassan, S.-U., Imran, M., Iqbal, S., Aljohani, N. R., & Nawaz, R. (2018). Deep context of citations using machine-learning models in scholarly full-text articles. Scientometrics, 117(3), 1645\u20131662. https:\/\/doi.org\/10.1007\/s11192-018-2944-y","journal-title":"Scientometrics"},{"issue":"4","key":"4003_CR16","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1017\/S1351324916000346","volume":"23","author":"M Hernandez-Alvarez","year":"2017","unstructured":"Hernandez-Alvarez, M., Soriano, J. M. G., & Martinez-Barco, P. (2017). Citation function, polarity and influence classification. Natural Language Engineering, 23(4), 561\u2013588. https:\/\/doi.org\/10.1017\/S1351324916000346","journal-title":"Natural Language Engineering"},{"key":"4003_CR17","first-page":"317","volume-title":"Biomedicine in the Twentieth Century: Practices, Policies, and Politics, volume 72 of Biomedical and Health Research","author":"JR Hollingsworth","year":"2008","unstructured":"Hollingsworth, J. R. (2008). Scientific discoveries: An institutionalist and path-dependent perspective. In C. Hannaway (Ed.), Biomedicine in the Twentieth Century: Practices, Policies, and Politics, volume 72 of Biomedical and Health Research. (pp. 317\u2013353). National Institutes of Health."},{"key":"4003_CR18","doi-asserted-by":"publisher","unstructured":"Huang, Y. H., Ko, M. T., Hsu, C. N. (2014). Identifying transformative research in biomedical sciences. In: S. M. Cheng, & M. Y. Day (Eds.), Technologies and applications of artificial intelligence. TAAI 2014. Lecture Notes in Computer Science, vol. 8916. Springer, Cham. https:\/\/doi.org\/10.1007\/978-3-319-13987-6_18","DOI":"10.1007\/978-3-319-13987-6_18"},{"key":"4003_CR19","doi-asserted-by":"publisher","unstructured":"Huang, Y., Hsu, C. & Lerman, K. (2013). Identifying transformative scientific research. In 2013 IEEE 13th international conference on data mining, Dallas, TX, USA (pp. 291\u2013300). https:\/\/doi.org\/10.1109\/ICDM.2013.120","DOI":"10.1109\/ICDM.2013.120"},{"key":"4003_CR20","volume-title":"The structure of scientific revolutions","author":"TS Kuhn","year":"1962","unstructured":"Kuhn, T. S. (1962). The structure of scientific revolutions. University of Chicago Press."},{"key":"4003_CR21","unstructured":"Nakov, P. I., Schwartz, A. S., & Hearst, M. A. (2004). Citances: Citation sentences for semantic analysis of bioscience text. In Proceedings of the SIGIR \u201804 workshop on Search and Discovery in Bioinformatics."},{"key":"4003_CR22","unstructured":"National Institutes of Health. NIH Director\u2019s Transformative Research Award. (2016). Retrieved December 23, 2019 from https:\/\/commonfund.nih.gov\/tra\/description"},{"key":"4003_CR23","unstructured":"National Science Board. (2007). Enhancing Support of Transformative Research at the National Science Foundation. Retrieved December 24, 2019 from https:\/\/www.nsf.gov\/nsb\/documents\/2007\/tr_report.pdf"},{"key":"4003_CR24","unstructured":"National Science Foundation. (2015). Definition of Transformative Research. Retrieved December 24, 2019 from https:\/\/www.nsf.gov\/about\/transformative_research\/definition.jsp"},{"key":"4003_CR25","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1016\/j.techfore.2012.09.017","volume":"81","author":"IV Ponomarev","year":"2014","unstructured":"Ponomarev, I. V., Williams, D. E., Hackett, C. J., Schnell, J. D., & Haak, L. L. (2014). Predicting highly cited papers: A method for early detection of candidate breakthroughs. Technological Forecasting and Social Change, 81, 49\u201355. https:\/\/doi.org\/10.1016\/j.techfore.2012.09.017","journal-title":"Technological Forecasting and Social Change"},{"key":"4003_CR26","unstructured":"Radev, D., & Abu-Jbara, A. (2012). Rediscovering ACL discoveries through the lens of ACL anthology network citing sentences. In Proceedings of the ACL-2012 Special Workshop on Rediscovering 50 Years of Discoveries, Jeju Island, Korea, jul 2012 (pp. 1\u201312): Association for Computational Linguistics"},{"issue":"1","key":"4003_CR27","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1016\/j.joi.2016.11.001","volume":"11","author":"H Small","year":"2017","unstructured":"Small, H., Tseng, H., & Patek, M. (2017). Discovering discoveries: Identifying biomedical discoveries using citation contexts. Journal of Informetrics, 11(1), 46\u201362. https:\/\/doi.org\/10.1016\/j.joi.2016.11.001","journal-title":"Journal of Informetrics"},{"key":"4003_CR28","unstructured":"Valenzuela, M., Ha, V., & Etzioni, O. (2015). Identifying Meaningful Citations. Paper presented at the AAAI Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence"},{"issue":"5","key":"4003_CR29","doi-asserted-by":"publisher","first-page":"1210","DOI":"10.1002\/asi.23505","volume":"67","author":"JJ Winnink","year":"2016","unstructured":"Winnink, J. J., Tijssen, R. J. W., & van Raan, A. F. J. (2016). Theory-changing breakthroughs in science: The impact of research teamwork on scientific discoveries. Journal of the Association for Information Science and Technology, 67(5), 1210\u20131223. https:\/\/doi.org\/10.1002\/asi.23505","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"4003_CR30","doi-asserted-by":"publisher","first-page":"673","DOI":"10.1016\/j.techfore.2018.05.018","volume":"146","author":"JJ Winnink","year":"2019","unstructured":"Winnink, J. J., Tijssen, R. J. W., & van Raan, A. F. J. (2019). Searching for new breakthroughs in science: How effective are computerised detection algorithms? Technological Forecasting and Social Change, 146, 673\u2013686. https:\/\/doi.org\/10.1016\/j.techfore.2018.05.018","journal-title":"Technological Forecasting and Social Change"},{"issue":"2","key":"4003_CR31","doi-asserted-by":"publisher","first-page":"807","DOI":"10.1007\/s11192-016-1861-1","volume":"107","author":"HN Wolcott","year":"2016","unstructured":"Wolcott, H. N., Fouch, M. J., Hsu, E. R., DiJoseph, L. G., Bernaciak, C. A., Corrigan, J. G., et al. (2016). Modeling time-dependent and -independent indicators to facilitate identification of breakthrough research papers. Scientometrics, 107(2), 807\u2013817. https:\/\/doi.org\/10.1007\/s11192-016-1861-1","journal-title":"Scientometrics"},{"key":"4003_CR32","unstructured":"Zhang, Y., & Wallace, B. (2015). A Sensitivity Analysis of (and Practitioners\u2019 Guide to) Convolutional Neural Networks for Sentence Classification. arXiv e-prints, http:\/\/arxiv.org\/abs\/1510.03820"},{"key":"4003_CR6","unstructured":"19th National Congress of the Communist Party of China. (2017). Secure a decisive victory in building a moderately prosperous society in all respects and strive for the great success of socialism with Chinese characteristics for a New Era. Retrieved December 03, 2019 from http:\/\/www.china.org.cn\/20171105-002.pdf"}],"container-title":["Scientometrics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-021-04003-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11192-021-04003-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-021-04003-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,24]],"date-time":"2021-06-24T14:31:51Z","timestamp":1624545111000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11192-021-04003-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,23]]},"references-count":32,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2021,7]]}},"alternative-id":["4003"],"URL":"https:\/\/doi.org\/10.1007\/s11192-021-04003-z","relation":{},"ISSN":["0138-9130","1588-2861"],"issn-type":[{"value":"0138-9130","type":"print"},{"value":"1588-2861","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5,23]]},"assertion":[{"value":"26 June 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 April 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 May 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}