{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T17:42:49Z","timestamp":1781113369048,"version":"3.54.1"},"reference-count":28,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2021,7,28]],"date-time":"2021-07-28T00:00:00Z","timestamp":1627430400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,7,28]],"date-time":"2021-07-28T00:00:00Z","timestamp":1627430400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Lang Resources &amp; Evaluation"],"published-print":{"date-parts":[[2022,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The extraction of information from Dutch archaeological grey literature has recently been investigated by the AGNES project. AGNES aims to disclose relevant information by means of a web search engine, to enable researchers to search through excavation reports. In this paper, we focus on the multi-labelling of archaeological excavation reports with time periods and site types, and provide a manually labelled reference set to this end. We propose a series of approaches, pre-processing methods, and various modifications of the training set to address the often low quality of both texts and labels. We find that despite those issues, our proposed methods lead to promising results.<\/jats:p>","DOI":"10.1007\/s10579-021-09552-6","type":"journal-article","created":{"date-parts":[[2021,7,28]],"date-time":"2021-07-28T10:02:48Z","timestamp":1627466568000},"page":"543-572","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Labelling the past: data set creation and multi-label classification of Dutch archaeological excavation reports"],"prefix":"10.1007","volume":"56","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1623-1340","authenticated-orcid":false,"given":"Alex","family":"Brandsen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Koole","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,7,28]]},"reference":[{"key":"9552_CR1","unstructured":"Branco, P., Torgo, L., & Ribeiro, R. (2015). A survey of predictive modelling under imbalanced distributions."},{"key":"9552_CR2","doi-asserted-by":"publisher","unstructured":"Brandsen, A., & Koole, M. (2020). Alexbrandsen\/archaeo-labelling-gold-standard: First version (Version v1.0.). Zenodo. https:\/\/doi.org\/10.5281\/zenodo.4115747","DOI":"10.5281\/zenodo.4115747"},{"issue":"1","key":"9552_CR3","doi-asserted-by":"publisher","first-page":"21","DOI":"10.5334\/jcaa.33","volume":"2","author":"A Brandsen","year":"2019","unstructured":"Brandsen, A., Lambers, K., Verberne, S., & Wansleeben, M. (2019). User requirement solicitation for an information retrieval system applied to Dutch grey literature in the archaeology domain. Journal of Computer Applications in Archaeology, 2(1), 21\u201330. https:\/\/doi.org\/10.5334\/jcaa.33","journal-title":"Journal of Computer Applications in Archaeology"},{"key":"9552_CR4","unstructured":"Brandsen, A., Verberne, S., Lambers, K., & Wansleeben, M. (2021). Usability evaluation for online professional search in the Dutch archaeology domain. arXiv. http:\/\/arxiv.org\/abs\/2103.04437"},{"key":"9552_CR5","doi-asserted-by":"publisher","unstructured":"Brandsen, A., Verberne, S., Wansleeben, M., & Lambers, K. (2020). Creating a dataset for named entity recognition in the archaeology domain. In Proceedings of The 12th Language Resources and Evaluation Conference (pp. 4573\u20134577). https:\/\/doi.org\/10.5281\/zenodo.3544544","DOI":"10.5281\/zenodo.3544544"},{"key":"9552_CR6","unstructured":"Brandt, R., Drenth, E., Montforts, M., Proos, R., Roorda, I., & Wiemer, R. (1992). Archeologisch Basisregister. Tech. rep., Rijksdienst voor Cultureel Erfgoed, Amersfoort."},{"key":"9552_CR7","doi-asserted-by":"publisher","first-page":"4","DOI":"10.19153\/cleiej.14.1.4","volume":"14","author":"EA Cherman","year":"2011","unstructured":"Cherman, E. A., Monard, M. C., & Metz, J. (2011). Multi-label problem transformation methods: A case study. CLEI Electronic Journal, 14, 4\u20134.","journal-title":"CLEI Electronic Journal"},{"key":"9552_CR22","unstructured":"de\u00a0Romas, R. (2019). Multi-label text classification for ground lease documents. Master\u2019s thesis, University of Amsterdam."},{"key":"9552_CR8","unstructured":"Devlin, J., Chang, M.W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of deep bidirectional transformers for language understanding. CoRR. http:\/\/arxiv.org\/abs\/1810.04805"},{"key":"9552_CR9","doi-asserted-by":"crossref","unstructured":"Fiorucci, M., Khoroshiltseva, M., Pontil, M., Traviglia, A., Del Bue, A., & James, S. (2020). Machine learning for cultural heritage: A survey. Pattern Recognition Letters, 133, 102\u2013108.","DOI":"10.1016\/j.patrec.2020.02.017"},{"key":"9552_CR10","unstructured":"Glyph & Cog LLC: pdftotext. (1996). https:\/\/www.xpdfreader.com\/pdftotext-man.html"},{"issue":"1","key":"9552_CR11","doi-asserted-by":"publisher","first-page":"18","DOI":"10.2478\/jdis-2020-0003","volume":"5","author":"K Golub","year":"2020","unstructured":"Golub, K., Hagelb\u00e4ck, J., & Ard\u00f6, A. (2020). Automatic classification of swedish metadata using dewey decimal classification: A comparison of approaches. Journal of Data and Information Science, 5(1), 18. https:\/\/doi.org\/10.2478\/jdis-2020-0003","journal-title":"Journal of Data and Information Science"},{"key":"9552_CR12","unstructured":"Honnibal, M., & Montani, I. (2017). spaCy 2: Natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing. To appear, still as of February 2020."},{"key":"9552_CR13","doi-asserted-by":"publisher","unstructured":"Jeffrey, S., Richards, J., Ciravegna, F., Waller, S., Chapman, S., & Zhang, Z. (2009). The archaeotools project: Faceted classification and natural language processing in an archaeological context. Philosophical transactions. Series A, Mathematical, physical, and engineering sciences, 367(1897), 2507\u20132519. https:\/\/doi.org\/10.1098\/rsta.2009.0038","DOI":"10.1098\/rsta.2009.0038"},{"key":"9552_CR14","doi-asserted-by":"crossref","unstructured":"Joachims, T. (1998). Text categorization with support vector machines: Learning with many relevant features. In Machine Learning: ECML-98 (pp. 137\u2013142). Springer.","DOI":"10.1007\/BFb0026683"},{"key":"9552_CR15","volume-title":"(Semi-) Automatic Cataloguing of Textual Cultural Heritage Objects","author":"M Kleppe","year":"2019","unstructured":"Kleppe, M., Hendrickx, I., Veldhoen, S., Brandsen, A., Vos, H. D., Goes, K., Huang, L., Huurdeman, H., Kim, A., Mesbah, S., Reuver, M., Wang, S., & Zijdeman, R. (2019). (Semi-) automatic cataloguing of textual cultural heritage objects. Tech. rep. KB (National Library of the Netherlands), Den Haag."},{"issue":"3","key":"9552_CR16","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1515\/jib-2011-177","volume":"8","author":"R Laza","year":"2011","unstructured":"Laza, R., Pav\u00f3n, R., Reboiro-Jato, M., & Fdez-Riverola, F. (2011). Evaluating the effect of unbalanced data in biomedical document classification. Journal of Integrative Bioinformatics, 8(3), 177. https:\/\/doi.org\/10.1515\/jib-2011-177","journal-title":"Journal of integrative bioinformatics"},{"key":"9552_CR17","doi-asserted-by":"publisher","unstructured":"Mohammed, R., Rawashdeh, J., & Abdullah, M. (2020). Machine learning with oversampling and undersampling techniques: Overview study and experimental results. In 2020 11th International Conference on Information and Communication Systems (ICICS) (pp. 243\u2013248). https:\/\/doi.org\/10.1109\/ICICS49469.2020.239556","DOI":"10.1109\/ICICS49469.2020.239556"},{"issue":"2","key":"9552_CR18","first-page":"1","volume":"7","author":"H Paijmans","year":"2010","unstructured":"Paijmans, H., & Brandsen, A. (2010). Searching in archaeological texts: Problems and solutions using an artificial intelligence approach. PalArch\u2019s Journal of Vertebrate Palaeontology, 7(2), 1\u20136.","journal-title":"PalArch\u2019s Journal of Vertebrate Palaeontology"},{"key":"9552_CR19","unstructured":"Postma, M., van Miltenburg, E., Segers, R., Schoen, A., & Vossen, P. (2016). Open Dutch WordNet. In Proceedings of the Eight Global Wordnet Conference (pp. 300\u2013308). Bucharest, Romania."},{"issue":"1","key":"9552_CR20","first-page":"37","volume":"2","author":"D Powers","year":"2011","unstructured":"Powers, D. (2011). Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation. International Journal of Machine Learning Technology, 2(1), 37\u201363.","journal-title":"International Journal of Machine Learning Technology"},{"key":"9552_CR21","unstructured":"Renfrew, C., & Bahn, P.G. (2019). Archaeology: Theories, methods and practice (8th ed.). Thames and Hudson."},{"key":"9552_CR23","unstructured":"Sasaki, Y. (2007). The truth of the F-measure. Tech. rep., School of Computer Science, University of Manchester, Manchester. https:\/\/www.toyota-ti.ac.jp\/Lab\/Denshi\/COIN\/people\/yutaka.sasaki\/F-measure-YS-26Oct07.pdf"},{"issue":"9","key":"9552_CR24","doi-asserted-by":"publisher","first-page":"750","DOI":"10.1111\/j.1749-818X.2010.00230.x","volume":"4","author":"C Sporleder","year":"2010","unstructured":"Sporleder, C. (2010). Natural language processing for cultural heritage domains. Language and Linguistics Compass, 4(9), 750\u2013768. https:\/\/doi.org\/10.1111\/j.1749-818X.2010.00230.x","journal-title":"Language and Linguistics Compass"},{"key":"9552_CR25","doi-asserted-by":"publisher","first-page":"222","DOI":"10.1504\/IJMSO.2012.050183","volume":"7","author":"A Vlachidis","year":"2012","unstructured":"Vlachidis, A., & Tudhope, D. (2012). A pilot investigation of information extraction in the semantic annotation of archaeological reports. International Journal of Metadata, Semantics and Ontologies, 7, 222\u2013235. https:\/\/doi.org\/10.1504\/IJMSO.2012.050183","journal-title":"International Journal of Metadata, Semantics and Ontologies"},{"issue":"5","key":"9552_CR26","doi-asserted-by":"publisher","first-page":"1138","DOI":"10.1002\/asi.23485","volume":"67","author":"A Vlachidis","year":"2016","unstructured":"Vlachidis, A., & Tudhope, D. (2016). A knowledge-based approach to information extraction for semantic interoperability in the archaeology domain. Journal of the Association for Information Science and Technology, 67(5), 1138\u20131152. https:\/\/doi.org\/10.1002\/asi.23485","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"9552_CR27","unstructured":"Vlachidis, A., Tudhope, D., Wansleeben, M., Azzopardi, J., Green, K., Xia, L., & Wright, H. (2017). D16.4: Final report on natural language processing. Tech. rep., ARIADNE. http:\/\/legacy.ariadne-infrastructure.eu\/wp-content\/uploads\/2019\/01\/D16.4_Final_Report_on_Natural_Language_Processing_Final.pdf"},{"key":"9552_CR28","doi-asserted-by":"crossref","unstructured":"Wei, J., & Zou, K. (2019). EDA: Easy data augmentation techniques for boosting performance on text classification tasks. CoRR. http:\/\/arxiv.org\/abs\/1901.11196","DOI":"10.18653\/v1\/D19-1670"}],"container-title":["Language Resources and Evaluation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10579-021-09552-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10579-021-09552-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10579-021-09552-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,11]],"date-time":"2022-06-11T10:08:15Z","timestamp":1654942095000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10579-021-09552-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,28]]},"references-count":28,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,6]]}},"alternative-id":["9552"],"URL":"https:\/\/doi.org\/10.1007\/s10579-021-09552-6","relation":{},"ISSN":["1574-020X","1574-0218"],"issn-type":[{"value":"1574-020X","type":"print"},{"value":"1574-0218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,28]]},"assertion":[{"value":"2 July 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 July 2021","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}