{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T21:22:04Z","timestamp":1775078524057,"version":"3.50.1"},"reference-count":77,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2020,6,9]],"date-time":"2020-06-09T00:00:00Z","timestamp":1591660800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,6,9]],"date-time":"2020-06-09T00:00:00Z","timestamp":1591660800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["GI-1259-1"],"award-info":[{"award-number":["GI-1259-1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Scientometrics"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Word embedding, which represents individual words with semantically fixed-length vectors, has made it possible to successfully apply deep learning to natural language processing tasks such as semantic role-modeling, question answering, and machine translation. As math text consists of natural text, as well as math expressions that similarly exhibit linear correlation and contextual characteristics, word embedding techniques can also be applied to math documents. However, while mathematics is a precise and accurate science, it is usually expressed through imprecise and less accurate descriptions, contributing to the relative dearth of machine learning applications for information retrieval in this domain. Generally, mathematical documents communicate their knowledge with an ambiguous, context-dependent, and non-formal language. Given recent advances in word embedding, it is worthwhile to explore their use and effectiveness in math information retrieval tasks, such as math language processing and semantic knowledge extraction. In this paper, we explore math embedding by testing it on several different scenarios, namely, (1) math-term similarity, (2) analogy, (3) numerical concept-modeling based on the centroid of the keywords that characterize a concept, (4) math search using query expansions, and (5) semantic extraction, i.e., extracting descriptive phrases for math expressions. Due to the lack of benchmarks, our investigations were performed using the arXiv collection of STEM documents and carefully selected illustrations on the Digital Library of Mathematical Functions (DLMF: NIST digital library of mathematical functions. Release 1.0.20 of 2018-09-1, 2018). Our results show that math embedding holds much promise for similarity, analogy, and search tasks. However, we also observed the need for more robust math embedding approaches. Moreover, we explore and discuss fundamental issues that we believe thwart the progress in mathematical information retrieval in the direction of machine learning.<\/jats:p>","DOI":"10.1007\/s11192-020-03502-9","type":"journal-article","created":{"date-parts":[[2020,6,9]],"date-time":"2020-06-09T12:03:04Z","timestamp":1591704184000},"page":"3017-3046","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["Math-word embedding in math search and semantic extraction"],"prefix":"10.1007","volume":"125","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5828-5497","authenticated-orcid":false,"given":"Andr\u00e9","family":"Greiner-Petter","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Abdou","family":"Youssef","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Terry","family":"Ruas","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bruce R.","family":"Miller","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Moritz","family":"Schubotz","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Akiko","family":"Aizawa","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bela","family":"Gipp","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2020,6,9]]},"reference":[{"key":"3502_CR1","unstructured":"Aizawa, A., Kohlhase, M., & Ounis, I. (2013). NTCIR-10 math pilot task overview (pp. 654\u2013661)."},{"key":"3502_CR2","unstructured":"Aizawa, A., Kohlhase, M., Ounis, I., & Schubotz, M. (2014). NTCIR-11 math-2 task overview. In Proceedings of the 11th NTCIR conference on evaluation of information access technologies, NTCIR-11, National Center of Sciences, Tokyo, Japan, December 9\u201312, 2014 (pp. 88\u201398). National Institute of Informatics (NII)."},{"key":"3502_CR3","unstructured":"Al-Tamimi, M., & Youssef, A. (2007). An extensive math query language. In The ISCA 16th international conference on software engineering and data engineering (SEDE-2007)."},{"key":"3502_CR4","doi-asserted-by":"crossref","unstructured":"Almasri, M., Berrut, C., & Chevallet, J.-P. (2016). A comparison of deep learning based query expansion with pseudo-relevance feedback and mutual information. In ECIR (pp. 709\u2013715).","DOI":"10.1007\/978-3-319-30671-1_57"},{"key":"3502_CR5","first-page":"1137","volume":"3","author":"Y Bengio","year":"2003","unstructured":"Bengio, Y., Ducharme, R., Vincent, P., & Janvin, C. (2003). A neural probabilistic language model. Journal of Machine Learning Research, 3, 1137\u20131155.","journal-title":"Journal of Machine Learning Research"},{"key":"3502_CR6","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1162\/tacl_a_00051","volume":"5","author":"P Bojanowski","year":"2017","unstructured":"Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017). Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135\u2013146.","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"3502_CR7","doi-asserted-by":"crossref","unstructured":"Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015). A large annotated corpus for learning natural language inference. In Proceedings of 2015 conference on empirical methods in natural language processing (EMNLP), Lisbon, Portugal (pp. 632\u2013642). ACL.","DOI":"10.18653\/v1\/D15-1075"},{"key":"3502_CR8","doi-asserted-by":"crossref","unstructured":"Camacho-Collados, J., Pilehvar, M. T., & Navigli, R. (2015). A unified multilingual semantic representation of concepts. In Proceedings of 53rd annual meeting of the association for computational linguistics (ACL), Beijing, China (pp. 741\u2013751). ACL.","DOI":"10.3115\/v1\/P15-1072"},{"key":"3502_CR9","doi-asserted-by":"crossref","unstructured":"Caselles-Dupr\u00e9, H., Lesaint, F., & Royo-Letelier, J. (2018). Word2vec applied to recommendation: Hyperparameters matter. In S. Pera, M. D. Ekstrand, X. Amatriain, & J. O\u2019Donovan (Eds.), Proceedings of the 12th ACM conference on recommender systems, RecSys 2018, Vancouver, BC, Canada, October 2\u20137, 2018 (pp. 352\u2013356). ACM.","DOI":"10.1145\/3240323.3240377"},{"key":"3502_CR10","doi-asserted-by":"crossref","unstructured":"Cer, D., Yang, Y., Kong, S., Hua, N., Limtiaco, N., John, R. S., Constant, N., Guajardo-Cespedes, M., Yuan, S., Tar, C., Strope, B., & Kurzweil, R. (2018). Universal sentence encoder for English. In Proceedings of conference on empirical methods in natural language processing (EMNLP), Brussels, Belgium (pp. 169\u2013174). ACL.","DOI":"10.18653\/v1\/D18-2029"},{"key":"3502_CR11","doi-asserted-by":"crossref","unstructured":"Chelba, C., Mikolov, T., Schuster, M., Ge, Q., Brants, T., Koehn, P., & Robinson, T. (2014). One billion word benchmark for measuring progress in statistical language modeling. In INTERSPEECH 2014, 15th annual conference of the international speech communication association, Singapore (pp. 2635\u20132639). ISCA.","DOI":"10.21437\/Interspeech.2014-564"},{"key":"3502_CR12","doi-asserted-by":"crossref","unstructured":"Chen, Q., Zhu, X., Ling, Z., Wei, S., Jiang, H., & Inkpen, D. (2017). Enhanced LSTM for natural language inference. In Proceedings of the 55th annual meeting of the association for computational linguistics (ACL), Vancouver, Canada, Volume 1: Long Papers (pp. 1657\u20131668). ACL.","DOI":"10.18653\/v1\/P17-1152"},{"key":"3502_CR13","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1162\/tacl_a_00104","volume":"4","author":"JPC Chiu","year":"2016","unstructured":"Chiu, J. P. C., & Nichols, E. (2016). Named entity recognition with bidirectional LSTM-CNNS. TACL, 4, 357\u2013370.","journal-title":"TACL"},{"key":"3502_CR14","doi-asserted-by":"crossref","unstructured":"Cho, K., van Merrienboer, B., G\u00fcl\u00e7ehre, \u00c7., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder\u2013decoder for statistical machine translation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), Doha, Qatar (pp. 1724\u20131734). ACL.","DOI":"10.3115\/v1\/D14-1179"},{"key":"3502_CR15","doi-asserted-by":"crossref","unstructured":"Clark, C., & Gardner, M. (2018). Simple and effective multi-paragraph reading comprehension. In Proceedings of the 56th annual meeting of the association for computational linguistics (ACL), Melbourne, Australia, Volume 1 (pp. 845\u2013855). ACL.","DOI":"10.18653\/v1\/P18-1078"},{"key":"3502_CR16","unstructured":"Croft, W. B., Metzler, D., & Strohman, T. (2009). Search engines\u2014Information retrieval in practice."},{"key":"3502_CR17","doi-asserted-by":"crossref","unstructured":"Dai, Z., Yang, Z., Yang, Y., Carbonell, J. G., Le, Q. V., & Salakhutdinov, R. (2019). Transformer-xl: Attentive language models beyond a fixed-length context. In Proceedings of the 57th conference of the association for computational linguistics (ACL), Florence, Italy, July 28\u2013August 2, 2019, Volume 1 (pp. 2978\u20132988). ACL.","DOI":"10.18653\/v1\/P19-1285"},{"key":"3502_CR18","unstructured":"Devlin, J., Chang, M., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: Human language technologies (NAACL-HLT), Minneapolis, MN, USA, June 2\u20137, 2019, Volume 1 (pp. 4171\u20134186). ACL."},{"key":"3502_CR19","doi-asserted-by":"crossref","unstructured":"Devlin, J., Zbib, R., Huang, Z., Lamar, T., Schwartz, R. M., & Makhoul, J. (2014). Fast and robust neural network joint models for statistical machine translation. In Proceedings of the 52nd annual meeting of the association for computational linguistics, ACL, Baltimore, MD, USA, Volume 1: Long Papers (pp. 1370\u20131380). ACL.","DOI":"10.3115\/v1\/P14-1129"},{"key":"3502_CR20","unstructured":"DLMF. (2018). NIST Digital Library of Mathematical Functions. http:\/\/dlmf.nist.gov\/, Release 1.0.20 of 2018-09-1. F.\u00a0W.\u00a0J. Olver, A.\u00a0B. Olde Daalhuis, D.\u00a0W. Lozier, B.\u00a0I. Schneider, R.\u00a0F. Boisvert, C.\u00a0W. Clark, B.\u00a0R. Miller, B.\u00a0V. Saunders, H.\u00a0S.\u00a0Cohl, & M.\u00a0A.\u00a0McClain (Eds)."},{"key":"3502_CR21","doi-asserted-by":"crossref","unstructured":"Elizarov, A., Kirillovich, A., Lipachev, E., & Nevzorova, O. (2017). Ontomath digital ecosystem: Ontologies, mathematical knowledge analytics and management. CoRR. arXiv:1702.05112.","DOI":"10.1007\/978-3-319-57135-5_3"},{"issue":"2","key":"3502_CR22","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1023\/A:1001181927857","volume":"32","author":"C Fellbaum","year":"1998","unstructured":"Fellbaum, C. (1998). A semantic network of English: The mother of all wordnets. Computers and the Humanities, 32(2), 209\u2013220.","journal-title":"Computers and the Humanities"},{"key":"3502_CR23","unstructured":"Gao, L., Jiang, Z., Yin, Y., Yuan, K., Yan, Z., & Tang, Z. (2017). Preliminary exploration of formula embedding for mathematical information retrieval: Can mathematical formulae be embedded like a natural language? CoRR. arXiv:1707.05154."},{"key":"3502_CR24","unstructured":"Ginev, D. (2018). arxmliv:08.2018 dataset, an html5 conversion of arxiv.org. SIGMathLing\u2014Special Interest Group on Math Linguistics."},{"key":"3502_CR25","unstructured":"Gong, Y., Luo, H., & Zhang, J. (2018). Natural language inference over interaction space. In 6th international conference on learning representations, ICLR 2018, Vancouver, BC, Canada, Conference Track Proceedings. OpenReview.net."},{"key":"3502_CR26","unstructured":"Greiner-Petter, A., Ruas, T., Schubotz, M., Aizawa, A., Grosky, W. I., & Gipp, B. (2019). Why machines cannot learn mathematics, yet. In Proceedings of 4th BIRNDL workshop at 42nd SIGIR, Paris, France (pp. 130\u2013137). CEUR-WS.org."},{"key":"3502_CR27","doi-asserted-by":"crossref","unstructured":"He, L., Lee, K., Lewis, M., & Zettlemoyer, L. (2017). Deep semantic role labeling: What works and what\u2019s next. In Proceedings of the 55th annual meeting of the association for computational linguistics, ACL 2017, Vancouver, Canada, July 30\u2013August 4, Volume 1: Long Papers (pp. 473\u2013483).","DOI":"10.18653\/v1\/P17-1044"},{"key":"3502_CR28","doi-asserted-by":"crossref","unstructured":"Iacobacci, I., Pilehvar, M. T., & Navigli, R. (2015). Sensembed: Learning sense embeddings for word and relational similarity. In Proceedings of 53rd annual meeting of the association for computational linguistics (ACL), Beijing, China (Vol. 1, pp. 95\u2013105). ACL.","DOI":"10.3115\/v1\/P15-1010"},{"key":"3502_CR29","doi-asserted-by":"crossref","unstructured":"Iacobacci, I., Pilehvar, M. T., & Navigli, R. (2016). Embeddings for word sense disambiguation: An evaluation study. In Proceedings of 54th annual meeting of the association for computational linguistics (ACL), Berlin, Germany (Vol. 1). ACL.","DOI":"10.18653\/v1\/P16-1085"},{"key":"3502_CR30","doi-asserted-by":"crossref","unstructured":"Kim, Y. (2014). Convolutional neural networks for sentence classification. In Proceedings of the 2014 conference on empirical methods in natural language processing, EMNLP 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL (pp. 1746\u20131751). ACL.","DOI":"10.3115\/v1\/D14-1181"},{"key":"3502_CR31","unstructured":"Kohlhase, M. (2017). Math object identifiers\u2014Towards research data in mathematics. In M. Leyer (Ed.), Lernen, Wissen, Daten, Analysen (LWDA) conference proceedings, Rostock, Germany, September 11\u201313, 2017, CEUR Workshop Proceedings (Vol. 1917, p. 241). CEUR-WS.org."},{"key":"3502_CR32","unstructured":"Krishna, A., Jin, Y., Foster, C., Youssef, A., & Gabel, G. (2019). Query expansion for patent searching using word embedding and professional crowdsourcing. In 2019 AAAI fall symposium series, FSS-19, Washington, DC. AAAI Press."},{"issue":"11\/12","key":"3502_CR33","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1045\/november14-kristianto","volume":"20","author":"GY Kristianto","year":"2014","unstructured":"Kristianto, G. Y., Topic, G., & Aizawa, A. (2014). Extracting textual descriptions of mathematical expressions in scientific papers. D-Lib Magazine, 20(11\/12), 9.","journal-title":"D-Lib Magazine"},{"issue":"2","key":"3502_CR34","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1007\/s10791-017-9296-8","volume":"20","author":"GY Kristianto","year":"2017","unstructured":"Kristianto, G. Y., Topic, G., & Aizawa, A. (2017). Utilizing dependency relationships between math expressions in math IR. Information Retrieval Journal, 20(2), 132\u2013167.","journal-title":"Information Retrieval Journal"},{"key":"3502_CR35","unstructured":"Krstovski, K., & Blei, D. M. (2018). Equation embeddings. CoRR. arXiv:1803.09123."},{"key":"3502_CR36","doi-asserted-by":"crossref","unstructured":"Kuzi, S., Shtok, A., & Kurland, O. (2016). Query expansion using word embeddings. In the 25th ACM international on conference on information and knowledge management (CIKM \u201916) (pp. 1929\u20131932).","DOI":"10.1145\/2983323.2983876"},{"key":"3502_CR37","doi-asserted-by":"crossref","unstructured":"Lau, J. H., & Baldwin, T. (2016). An empirical evaluation of doc2vec with practical insights into document embedding generation. In P. Blunsom, K. Cho, S. B. Cohen, E. Grefenstette, K. M. Hermann, L. Rimell, J. Weston, & S. W. Yih (Eds.), Proceedings of the 1st workshop on representation learning for NLP, Rep4NLP@ACL 2016, Berlin, Germany, August 11, 2016 (pp. 78\u201386). ACL.","DOI":"10.18653\/v1\/W16-1609"},{"key":"3502_CR38","unstructured":"Le, Q., & Mikolov, T. (2014). Distributed representations of sentences and documents. In Proceedings of the 31st international conference on machine learning (ICML), Beijing, China (pp. II\u20131188\u2013II\u20131196). JMLR.org."},{"key":"3502_CR39","doi-asserted-by":"crossref","unstructured":"Lee, K., He, L., Lewis, M., & Zettlemoyer, L. (2017). End-to-end neural coreference resolution. In Proceedings of the 2017 conference on empirical methods in natural language processing (EMNLP), Copenhagen, Denmark (pp. 188\u2013197). ACL.","DOI":"10.18653\/v1\/D17-1018"},{"key":"3502_CR40","doi-asserted-by":"crossref","unstructured":"Li, J., & Jurafsky, D. (2015). Do multi-sense embeddings improve natural language understanding? In Proceedings of conference on empirical methods in natural language processing (EMNLP), Lisbon, Portugal (pp. 1722\u20131732). ACL.","DOI":"10.18653\/v1\/D15-1200"},{"key":"3502_CR41","doi-asserted-by":"crossref","unstructured":"Liu, X., Shen, Y., Duh, K., & Gao, J. (2018). Stochastic answer networks for machine reading comprehension. In Proceedings of the 56th annual meeting of the association for computational linguistics (ACL), Melbourne, Australia, July 15\u201320, 2018, Volume 1 (pp. 1694\u20131704). ACL.","DOI":"10.18653\/v1\/P18-1157"},{"key":"3502_CR42","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., et al. (2019). Roberta: A robustly optimized BERT pretraining approach. CoRR. arXiv:1907.11692."},{"key":"3502_CR43","doi-asserted-by":"crossref","unstructured":"Mancini, M., Camacho-Collados, J., Iacobacci, I., & Navigli, R. (2017). Embedding words and senses together via joint know-ledge-enhanced training. In Proceedings of 21st conference on computational natural language learning (CoNLL), Vancouver, Canada (pp. 100\u2013111). ACL.","DOI":"10.18653\/v1\/K17-1012"},{"key":"3502_CR44","doi-asserted-by":"crossref","unstructured":"Matsuzaki, T., Iwane, H., Anai, H., & Arai, N. H. (2014). The most uncreative examinee: A first step toward wide coverage natural language math problem solving. In C. E. Brodley & P. Stone (Eds.), Proceedings of twenty-eighth AAAI conference on artificial intelligence, Qu\u00e9bec City, Qu\u00e9bec, Canada (pp. 1098\u20131104). AAAI Press.","DOI":"10.1609\/aaai.v28i1.8869"},{"key":"3502_CR45","unstructured":"Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013a). Efficient estimation of word representations in vector space. CoRR. arXiv:1301.3781."},{"key":"3502_CR46","unstructured":"Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013b). Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th international conference on neural information processing systems (NIPS), Volume 2 (pp. 3111\u20133119). USA: Curran Associates Inc."},{"issue":"11","key":"3502_CR47","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1145\/219717.219748","volume":"38","author":"GA Miller","year":"1995","unstructured":"Miller, G. A. (1995). Wordnet: A lexical database for english. Communications of the ACM, 38(11), 39\u201341.","journal-title":"Communications of the ACM"},{"key":"3502_CR48","doi-asserted-by":"crossref","unstructured":"Neelakantan, A., Shankar, J., Passos, A., & McCallum, A. (2014). Efficient non-parametric estimation of multiple embeddings per word in vector space. In Proceedings of conference on empirical methods in natural language processing (EMNLP), Doha, Qatar (pp. 1059\u20131069). ACL.","DOI":"10.3115\/v1\/D14-1113"},{"key":"3502_CR49","unstructured":"Pagel, R., & Schubotz, M. (2014). Mathematical language processing project. In Joint proceedings of the MathUI, OpenMath and ThEdu workshops and work in progress track at CICM co-located with conferences on intelligent computer mathematics (CICM 2014), Coimbra, Portugal, CEUR Workshop Proceedings (Vol. 1186). CEUR-WS.org."},{"issue":"1","key":"3502_CR50","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1162\/0891201053630264","volume":"31","author":"M Palmer","year":"2005","unstructured":"Palmer, M., Kingsbury, P., & Gildea, D. (2005). The proposition bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1), 71\u2013106.","journal-title":"Computational Linguistics"},{"key":"3502_CR51","doi-asserted-by":"crossref","unstructured":"Pennington, J., Socher, R., & Manning, C. D. (2014). Glove: Global vectors for word representation. In Proceedings of conference on empirical methods in natural language processing (EMNLP), Doha, Qatar, Volume 14 (pp. 1532\u20131543). ACL.","DOI":"10.3115\/v1\/D14-1162"},{"key":"3502_CR52","doi-asserted-by":"crossref","unstructured":"Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. In Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: Human language technologies (NAACL-HLT), New Orleans, Louisiana, USA, June 1\u20136, 2018, Volume 1 (pp. 2227\u20132237). ACL.","DOI":"10.18653\/v1\/N18-1202"},{"key":"3502_CR53","doi-asserted-by":"crossref","unstructured":"Pilehvar, M. T., Camacho-Collados, J., Navigli, R., & Collier, N. (2017). Towards a seamless integration of word senses into downstream NLP applications. CoRR. arXiv:1710.06632.","DOI":"10.18653\/v1\/P17-1170"},{"key":"3502_CR54","doi-asserted-by":"crossref","unstructured":"Pilehvar, M. T., & Collier, N. (2016). De-conflated semantic representations. In Proceedings of conference on empirical methods in natural language processing (EMNLP), Austin, Texas, USA (pp. 1680\u20131690). ACL.","DOI":"10.18653\/v1\/D16-1174"},{"key":"3502_CR55","doi-asserted-by":"crossref","unstructured":"Raganato, A., Bovi, C. D., & Navigli, R. (2017). Neural sequence learning models for word sense disambiguation. In Proceedings of the 2017 conference on empirical methods in natural language processing (EMNLP), Copenhagen, Denmark, September 9\u201311, 2017 (pp. 1156\u20131167). ACL.","DOI":"10.18653\/v1\/D17-1120"},{"key":"3502_CR56","doi-asserted-by":"crossref","unstructured":"Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 conference on empirical methods in natural language processing (EMNLP), Austin, Texas, USA (pp. 2383\u20132392). ACL.","DOI":"10.18653\/v1\/D16-1264"},{"key":"3502_CR57","unstructured":"\u0158eh\u016f\u0159ek, R., & Sojka, P. (2010). Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 workshop on new challenges for NLP frameworks, Valletta, Malta (pp. 45\u201350). ELRA. http:\/\/is.muni.cz\/publication\/884893\/en."},{"key":"3502_CR58","first-page":"313","volume-title":"The SMART retrieval system\u2014Experiments in automatic document processing","author":"J Rocchio","year":"1971","unstructured":"Rocchio, J. (1971). Relevance feedback in information retrieval. In The SMART retrieval system\u2014Experiments in automatic document processing (pp. 313\u2013323). Upper Saddle River: Prentice-Hall Inc."},{"issue":"1","key":"3502_CR59","doi-asserted-by":"publisher","first-page":"288","DOI":"10.1016\/j.eswa.2019.06.026","volume":"136","author":"T Ruas","year":"2019","unstructured":"Ruas, T., Grosky, W., & Aizawa, A. (2019). Multi-sense embeddings through a word sense disambiguation process. Expert Systems with Applications, 136(1), 288\u2013303.","journal-title":"Expert Systems with Applications"},{"key":"3502_CR77","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1016\/j.ins.2020.04.048","volume":"532","author":"T Ruas","year":"2020","unstructured":"Ruas T, Ferreira CHP, Grosky W, de Fran\u00e7a FO, de Medeiros DMR (2020) Enhanced word embeddings using multi-semantic representation through lexical chains. Information Sciences, 532, 16\u201332.","journal-title":"Information Sciences"},{"key":"3502_CR60","unstructured":"Rudolph, M. R., Ruiz, F. J. R., Athey, S., & Blei, D. M. (2017). Structured embedding models for grouped data. In Advances in neural information processing systems 30: Annual conference on neural information processing systems 2017, Long Beach, CA, USA (pp. 251\u2013261). Curran Associates, Inc."},{"key":"3502_CR61","doi-asserted-by":"crossref","unstructured":"Schubotz, M., Greiner-Petter, A., Scharpf, P., Meuschke, N., Cohl, H. S., & Gipp, B. (2018). Improving the representation and conversion of mathematical formulae by considering their textual context. In Proceedings of the 18th ACM\/IEEE on joint conference on digital libraries (JCDL), Fort Worth, TX, USA, June 03\u201307, 2018 (pp. 233\u2013242). ACM.","DOI":"10.1145\/3197026.3197058"},{"key":"3502_CR62","doi-asserted-by":"crossref","unstructured":"Schubotz, M., Grigorev, A., Leich, M., Cohl, H. S., Meuschke, N., Gipp, B., Youssef, A. S., & Markl, V. (2016). Semantification of identifiers in mathematics for better math information retrieval. In Proceedings of the 39th international ACM SIGIR conference on research and development in information retrieval (SIGIR), Pisa, Tuscany, Italy (pp. 135\u2013144). ACM.","DOI":"10.1145\/2911451.2911503"},{"key":"3502_CR63","doi-asserted-by":"crossref","unstructured":"Schubotz, M., Kr\u00e4mer, L., Meuschke, N., Hamborg, F., & Gipp, B. (2017). Evaluating and improving the extraction of mathematical identifier definitions. In Experimental IR meets multilinguality, multimodality, and interaction\u20148th international conference of the CLEF association, CLEF 2017, Dublin, Ireland, Lecture notes in computer science (Vol. 10456, pp. 82\u201394). Berlin: Springer.","DOI":"10.1007\/978-3-319-65813-1_7"},{"key":"3502_CR64","unstructured":"Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, Seattle, Washington, USA (pp. 1631\u20131642). ACL."},{"key":"3502_CR65","volume-title":"Query expansion for information retrieval","author":"O Vechtomova","year":"2009","unstructured":"Vechtomova, O. (2009). Query expansion for information retrieval. Boston, MA: Springer."},{"key":"3502_CR66","doi-asserted-by":"crossref","unstructured":"Wattenberg, M., Vi\u00e9gas, F., & Johnson, I. (2016). How to use t-SNE effectively. Distill.","DOI":"10.23915\/distill.00002"},{"key":"3502_CR67","doi-asserted-by":"crossref","unstructured":"Wiseman, S., Rush, A. M., & Shieber, S. M. (2016). Learning global features for coreference resolution. In NAACL HLT 2016, The 2016 conference of the North American chapter of the association for computational linguistics: Human language technologies, San Diego California, USA (pp. 994\u20131004). ACL.","DOI":"10.18653\/v1\/N16-1114"},{"key":"3502_CR68","first-page":"119","volume-title":"Towards a digital mathematics library","author":"M Wolska","year":"2010","unstructured":"Wolska, M., & Grigore, M. (2010). Symbol declarations in mathematical writing. In P. Sojka (Ed.), Towards a digital mathematics library (pp. 119\u2013127). Brno: Masaryk University Press."},{"key":"3502_CR69","unstructured":"Yang, Z., Dai, Z., Yang, Y., Carbonell, J. G., Salakhutdinov, R., & Le, Q. V. (2019). Xlnet: Generalized autoregressive pretraining for language understanding. In H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d\u2019Alch\u00e9-Buc, E. B. Fox, & R. Garnett (Eds.), Advances in neural information processing systems 32: Annual conference on neural information processing systems 2019, NeurIPS 2019, 8\u201314 December 2019, Vancouver, BC, Canada (pp. 5754\u20135764)."},{"key":"3502_CR70","doi-asserted-by":"crossref","unstructured":"Yasunaga, M., & Lafferty, J. (2019). Topiceq: A joint topic and mathematical equation model for scientific texts. CoRR. arXiv:1902.06034.","DOI":"10.1609\/aaai.v33i01.33017394"},{"key":"3502_CR71","unstructured":"Youssef, A. (2005). Search of mathematical contents: Issues and methods. In The ISCA 14th international conference on intelligent and adaptive systems and software engineering (IASSE-2005)."},{"key":"3502_CR72","doi-asserted-by":"crossref","unstructured":"Youssef, A. (2017). Part-of-math tagging and applications. In Intelligent computer mathematics (pp. 356\u2013374). Cham: Springer.","DOI":"10.1007\/978-3-319-62075-6_25"},{"key":"3502_CR73","doi-asserted-by":"crossref","unstructured":"Youssef, A., & Miller, B. R. (2019). Explorations into the use of word embedding in math search and math semantics. In Intelligent computer mathematics\u201412th international conference, CICM 2019, Prague, Czech Republic (pp. 291\u2013305). Springer.","DOI":"10.1007\/978-3-030-23250-4_20"},{"key":"3502_CR74","unstructured":"Zanibbi, R., Aizawa, A., Kohlhase, M., Ounis, I., Topic, G., & Davila, K. (2016a). NTCIR-12 MathIR task overview. In Proceedings of the 12th NTCIR conference on evaluation of information access technologies, National Center of Sciences, Tokyo, Japan. National Institute of Informatics (NII)."},{"key":"3502_CR75","doi-asserted-by":"crossref","unstructured":"Zanibbi, R., Davila, K., Kane, A., & Tompa, F. W. (2016b). Multi-stage math formula search: Using appearance-based similarity metrics at scale. In R. Perego, F. Sebastiani, J. A. Aslam, I. Ruthven, & J. Zobel (Eds.), Proceedings of 39th international ACM SIGIR conference on research and development in information retrieval, Pisa, Italy (pp. 145\u2013154). ACM.","DOI":"10.1145\/2911451.2911512"},{"key":"3502_CR76","doi-asserted-by":"crossref","unstructured":"Zhou, J., & Xu, W. (2015). End-to-end learning of semantic role labeling using recurrent neural networks. In Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing of the Asian federation of natural language processing (ACL), Beijing, China, Volume 1 (pp. 1127\u20131137). ACL.","DOI":"10.3115\/v1\/P15-1109"}],"container-title":["Scientometrics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-020-03502-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11192-020-03502-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-020-03502-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,27]],"date-time":"2022-10-27T10:28:47Z","timestamp":1666866527000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11192-020-03502-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,9]]},"references-count":77,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["3502"],"URL":"https:\/\/doi.org\/10.1007\/s11192-020-03502-9","relation":{},"ISSN":["0138-9130","1588-2861"],"issn-type":[{"value":"0138-9130","type":"print"},{"value":"1588-2861","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,9]]},"assertion":[{"value":"9 June 2020","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}