{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,3]],"date-time":"2026-05-03T14:50:39Z","timestamp":1777819839773,"version":"3.51.4"},"reference-count":13,"publisher":"Walter de Gruyter GmbH","issue":"4","license":[{"start":{"date-parts":[[2019,11,1]],"date-time":"2019-11-01T00:00:00Z","timestamp":1572566400000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec id=\"j_jdis-2019-0019_s_007\">\n                    <jats:title>Purpose<\/jats:title>\n                    <jats:p>To uncover the evaluation information on the academic contribution of research papers cited by peers based on the content cited by citing papers, and to provide an evidence-based tool for evaluating the academic value of cited papers.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"j_jdis-2019-0019_s_008\">\n                    <jats:title>Design\/methodology\/approach<\/jats:title>\n                    <jats:p>CiteOpinion uses a deep learning model to automatically extract citing sentences from representative citing papers; it starts with an analysis on the citing sentences, then it identifies major academic contribution points of the cited paper, positive\/negative evaluations from citing authors and the changes in the subjects of subsequent citing authors by means of Recognizing Categories of Moves (problems, methods, conclusions, etc.), and sentiment analysis and topic clustering.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"j_jdis-2019-0019_s_009\">\n                    <jats:title>Findings<\/jats:title>\n                    <jats:p>Citing sentences in a citing paper contain substantial evidences useful for academic evaluation. They can also be used to objectively and authentically reveal the nature and degree of contribution of the cited paper reflected by citation, beyond simple citation statistics.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"j_jdis-2019-0019_s_010\">\n                    <jats:title>Practical implications<\/jats:title>\n                    <jats:p>The evidence-based evaluation tool CiteOpinion can provide an objective and in-depth academic value evaluation basis for the representative papers of scientific researchers, research teams, and institutions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"j_jdis-2019-0019_s_011\">\n                    <jats:title>Originality\/value<\/jats:title>\n                    <jats:p>No other similar practical tool is found in papers retrieved.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"j_jdis-2019-0019_s_012\">\n                    <jats:title>Research limitations<\/jats:title>\n                    <jats:p>There are difficulties in acquiring full text of citing papers. There is a need to refine the calculation based on the sentiment scores of citing sentences. Currently, the tool is only used for academic contribution evaluation, while its value in policy studies, technical application, and promotion of science is not yet tested.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2478\/jdis-2019-0019","type":"journal-article","created":{"date-parts":[[2019,12,29]],"date-time":"2019-12-29T04:31:11Z","timestamp":1577593871000},"page":"26-41","source":"Crossref","is-referenced-by-count":8,"title":["CiteOpinion: Evidence-based Evaluation Tool for Academic Contributions of Research Papers Based on Citing Sentences"],"prefix":"10.2478","volume":"4","author":[{"given":"Xiaoqiu","family":"Le","sequence":"first","affiliation":[{"name":"National Science Library, Chinese Academy of Sciences , Beijing 100190 , China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jingdan","family":"Chu","sequence":"additional","affiliation":[{"name":"University of Chinese Academy of Sciences , Beijing 100190 , China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Siyi","family":"Deng","sequence":"additional","affiliation":[{"name":"University of Chinese Academy of Sciences , Beijing 100190 , China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qihang","family":"Jiao","sequence":"additional","affiliation":[{"name":"University of Chinese Academy of Sciences , Beijing 100190 , China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jingjing","family":"Pei","sequence":"additional","affiliation":[{"name":"University of Chinese Academy of Sciences , Beijing 100190 , China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Liya","family":"Zhu","sequence":"additional","affiliation":[{"name":"National Science Library, Chinese Academy of Sciences , Beijing 100190 , China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Junliang","family":"Yao","sequence":"additional","affiliation":[{"name":"University of Chinese Academy of Sciences , Beijing 100190 , China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"374","published-online":{"date-parts":[[2019,12,27]]},"reference":[{"key":"2026042921171306621_j_jdis-2019-0019_ref_001","doi-asserted-by":"crossref","unstructured":"Brooks, T.A. (1985). Private acts and public objects: An investigation of citer motivations. Journal of the American Society for Information Science, 36(4), 223\u2013229.","DOI":"10.1002\/asi.4630360402"},{"key":"2026042921171306621_j_jdis-2019-0019_ref_002","unstructured":"Garfield, E. (1964). Can citation indexing be automated? In the Proceedings of the Symposyumon Statistical Association, Washington DC, 8490. http:\/\/garfield.library.upenn.edu\/essays\/V1p084y1962-73.pdf"},{"key":"2026042921171306621_j_jdis-2019-0019_ref_003","doi-asserted-by":"crossref","unstructured":"Howard, J., & Ruder, S. (2018). Universal language model fine-tuning for text classification. In the Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. https:\/\/arxiv.org\/pdf\/1801.06146.pdf","DOI":"10.18653\/v1\/P18-1031"},{"key":"2026042921171306621_j_jdis-2019-0019_ref_004","unstructured":"Lei, S.W., Chen, H.H., Huang, Y., & Lu, W. (2016). Research on automatic recognition of academic citation context. Library and Information Service, 60(17), 78\u201387. DOI:10.13266\/j.issn.0252-3116.2016.17.012"},{"key":"2026042921171306621_j_jdis-2019-0019_ref_005","doi-asserted-by":"crossref","unstructured":"Liu, S.B., & Chen, C.M. (2013). The differences between latent topics in abstracts and citation contexts of citing papers. Journal of the American Society for Information Science and Technology, 64(3), 627\u2013639.","DOI":"10.1002\/asi.22771"},{"key":"2026042921171306621_j_jdis-2019-0019_ref_006","unstructured":"Liu, S.B., Wang, B., Tang, D.L., Ma, X., & Ding, K. (2015). Research on paper influence based on citation context: A case study of the Nobel Prize winner\u2019s paper. Library and Information Service, 59(24), 109\u2013114."},{"key":"2026042921171306621_j_jdis-2019-0019_ref_007","unstructured":"Liu Y., & Cui L.(2014). The information value of citation context in paper content analysis. Library and Information Service, 58(6), 101\u2013104."},{"key":"2026042921171306621_j_jdis-2019-0019_ref_008","unstructured":"Ma, F., & Wu, Y.S. (2009). A survey study on motivations for citation: A case study on periodicals research and library and information science community in China. Chinese Journal of Library and Information Science, 2(3), 28\u201343."},{"key":"2026042921171306621_j_jdis-2019-0019_ref_009","doi-asserted-by":"crossref","unstructured":"Mohammad, S., Dorr, B., Egan, M., Hassan, A., & Zajic, D.M. (2009). Using citations to generate surveys of scientific paradigms. In the Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Boulder: Association for Computational Linguistics, 584\u2013592.","DOI":"10.3115\/1620754.1620839"},{"key":"2026042921171306621_j_jdis-2019-0019_ref_010","unstructured":"Pei, J.J., & Le, X.Q. (2019). Identifying coordinate text blocks in discourses. Data Analysis and Knowledge Discovery, 3(5), 51\u201356. DOI: 10.11925\/infotech.2096-3467.2018.1380"},{"key":"2026042921171306621_j_jdis-2019-0019_ref_011","doi-asserted-by":"crossref","unstructured":"Small, H. (2011). Interpreting maps of science using citation context sentiments: A preliminary investigation. Scientometrics, 87(2), 373\u2013388.","DOI":"10.1007\/s11192-011-0349-2"},{"key":"2026042921171306621_j_jdis-2019-0019_ref_012","doi-asserted-by":"crossref","unstructured":"Spiegel-R\u00d6sing, I. (1977). Science studies: Bibliometric and content analysis. Social Studies of Science, 7(1), 97\u2013113.","DOI":"10.1177\/030631277700700111"},{"key":"2026042921171306621_j_jdis-2019-0019_ref_013","doi-asserted-by":"crossref","unstructured":"Teufel, S., Siddharthan, A., & Tidhar, D. (2006). Automatic classification of citation function. In the Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. Sydney: Association for Computational Linguistics, 103\u2013110.","DOI":"10.3115\/1610075.1610091"}],"container-title":["Journal of Data and Information Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/content.sciendo.com\/view\/journals\/jdis\/4\/4\/article-p26.xml","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.2478\/jdis-2019-0019\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.2478\/jdis-2019-0019\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T21:29:20Z","timestamp":1777498160000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.2478\/jdis-2019-0019\/html"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,1]]},"references-count":13,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2019,12,27]]},"published-print":{"date-parts":[[2019,11,1]]}},"alternative-id":["10.2478\/jdis-2019-0019"],"URL":"https:\/\/doi.org\/10.2478\/jdis-2019-0019","relation":{},"ISSN":["2543-683X"],"issn-type":[{"value":"2543-683X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,1]]}}}