{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T00:58:26Z","timestamp":1740099506833,"version":"3.37.3"},"publisher-location":"Cham","reference-count":63,"publisher":"Springer International Publishing","isbn-type":[{"type":"print","value":"9783030229474"},{"type":"electronic","value":"9783030229481"}],"license":[{"start":{"date-parts":[[2019,1,1]],"date-time":"2019-01-01T00:00:00Z","timestamp":1546300800000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019]]},"DOI":"10.1007\/978-3-030-22948-1_15","type":"book-chapter","created":{"date-parts":[[2019,8,13]],"date-time":"2019-08-13T05:02:59Z","timestamp":1565672579000},"page":"365-387","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Multilingual Patent Text Retrieval Evaluation: CLEF\u2013IP"],"prefix":"10.1007","author":[{"given":"Florina","family":"Piroi","sequence":"first","affiliation":[]},{"given":"Allan","family":"Hanbury","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2019,8,14]]},"reference":[{"issue":"4","key":"15_CR1","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1016\/S0172-2190(00)00073-9","volume":"22","author":"S Adams","year":"2000","unstructured":"Adams S (2000) Using the International Patent Classification in an online environment. World Patent Information 22(4):291\u2013300","journal-title":"World Patent Information"},{"key":"15_CR2","doi-asserted-by":"crossref","unstructured":"Adams SR (2011) Information sources in patents, 3rd edn. K.G. Saur, Munich","DOI":"10.1515\/9783110235128"},{"key":"15_CR3","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/978-3-642-19231-9_1","volume-title":"Current Challenges in Patent Information Retrieval","author":"Doreen Alberts","year":"2011","unstructured":"Alberts D, Yang CB, Fobare-DePonio D, Koubek K, Robins S, Rodgers M, Simmons E, DeMarco D (2011) Introduction to patent searching. In: Lupu M, Mayer K, Tait J, Trippe AJ (eds) Current challenges in patent information retrieval, the information retrieval series, vol 29. Springer, Berlin, pp 3\u201343. http:\/\/dx.doi.org\/10.1007\/978-3-642-19231-9_1"},{"key":"15_CR4","unstructured":"Anthon C (1841) A classical dictionary: containing an account of the principal proper names mentioned in ancient authors, and intended to elucidate all the important points connected with the geography, history, biography, mythology, and fine arts of the Greeks and Romans together with an account of coins, weights, and measures, with tabular values of the same. Harper & Bros"},{"issue":"3","key":"15_CR5","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1145\/322017.322021","volume":"24","author":"R Attar","year":"1977","unstructured":"Attar R, Fraenkel AS (1977) Local feedback in full-text retrieval systems. J ACM 24(3):397\u2013417. http:\/\/doi.acm.org\/10.1145\/322017.322021","journal-title":"J ACM"},{"key":"15_CR6","doi-asserted-by":"crossref","unstructured":"Becks D, Womser-Hacker C, Mandl T, K\u00f6lle R (2009) Patent retrieval experiments in the context of the CLEF IP Track 2009. In: Borri F, Nardi A, Peters C, Ferro N (eds) (2009) CLEF 2009 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1175\/","DOI":"10.1007\/978-3-642-15754-7_59"},{"key":"15_CR7","unstructured":"Becks D, Mandl T, Womser-Hacker C (2010) Phrases or terms? The impact of different query types. In: Braschler M, Harman DK, Pianta E, Ferro N (eds) (2010) CLEF 2010 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1176\/"},{"key":"15_CR8","unstructured":"Correa S, Buscaldi D, Rosso P (2009) NLEL-MAAT at CLEF-IP. In: Borri F, Nardi A, Peters C, Ferro N (eds) (2009) CLEF 2009 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1175\/"},{"key":"15_CR9","unstructured":"Derieux F, Bobeica M, Pois D, Raysz JP (2010) Combining semantics and statistics for patent classification. In: Braschler M, Harman DK, Pianta E, Ferro N (eds) (2010) CLEF 2010 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1176\/"},{"key":"15_CR10","unstructured":"D\u2019hondt E, Verberne S, Alink W, Cornacchia R (2011) Combining document representations for prior-art retrieval. In: Petras V, Forner P, Clough P, Ferro N (eds) (2011) CLEF 2011 Working Notes, CEUR Workshop Proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1177\/"},{"key":"15_CR11","unstructured":"Eiselt A, Oberreuter G (2013) The simpler the better - Retrieval Moderl comparison for Prior-Art Search in Patents at CLEF-IP 2013. In: Forner P, Navigli R, Tufis D, Ferro N (eds) (2013) CLEF 2013 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1179\/"},{"key":"15_CR12","unstructured":"EPO (1973) European Patent Convention (EPC), Implementing Regulations, Examination Procedure. http:\/\/www.epo.org\/law-practice\/legal-texts\/html\/epc\/2016\/e\/r71.html . Accessed Dec 2018"},{"issue":"2","key":"15_CR13","first-page":"1","volume":"5","author":"EPO","year":"2018","unstructured":"EPO (2018) Guidelines for Examination in the European Patent Office. Directorate Patent Law 5.2.1. http:\/\/www.epo.org\/law-practice\/legal-texts\/guidelines.html . Accessed Dec 2018","journal-title":"Directorate Patent Law"},{"key":"15_CR14","unstructured":"Frietsch R, Schmoch U, Looy B, Walsh P, Devroede R, Du Plessis M, Jung T, Meng Y, Neuh\u00e4usler P, Peeters B, Schubert T (2010) The value and indicator function of patents. Expertenkommission Forschung und Innovation (EFI) Studien zum deutschen Innovationssystem 15(15-2010)"},{"key":"15_CR15","unstructured":"Galasso A, Schankerman M (2013) Do patents help or hinder innovation? World Economic Forum. https:\/\/www.weforum.org\/agenda\/2013\/05\/do-patents-help-or-hinder-innovation . Accessed Dec 2018"},{"key":"15_CR16","unstructured":"Giachanou A, Salampasis M, Satratzemi M, Samaras N (2013) Report on the CLEF-IP 2013 experiments: multilayer collection selection on topically organized patents. In: Forner P, Navigli R, Tufis D, Ferro N (eds) (2013) CLEF 2013 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1179\/"},{"key":"15_CR17","unstructured":"Gobeill J, Ruch P (2012) BiTeM site report for the Claims to Passage task in CLEF-IP 2012. In: Forner P, Karlgren J, Womser-Hacker C, Ferro N (eds) (2012) CLEF 2012 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1178\/"},{"key":"15_CR18","unstructured":"Gobeill J, Theodoro D, Ruch P (2009) Exploring a wide range of simple pre and post processing strategies for patent searching in CLEF IP 2009. In: Borri F, Nardi A, Peters C, Ferro N (eds) (2009) CLEF 2009 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1175\/"},{"key":"15_CR19","unstructured":"Graf E, Azzopardi L (2008) A methodology for building a patent test collection for prior art search. In: Proceedings of the second international workshop on evaluating information access (EVIA)"},{"key":"15_CR20","unstructured":"Graf E, Azzopardi L, van Rijsbergen K (2009) Automatically generating queries for prior art search. In: Borri F, Nardi A, Peters C, Ferro N (eds) (2009) CLEF 2009 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1175\/"},{"key":"15_CR21","unstructured":"Guyot J, Benzineb K, Falquet G (2010) myClass: a mature tool for patent classification. In: Braschler M, Harman DK, Pianta E, Ferro N (eds) (2010) CLEF 2010 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1176\/"},{"key":"15_CR22","first-page":"1","volume-title":"Patents","author":"BH Hall","year":"2017","unstructured":"Hall BH (2017) Patents. Palgrave, Macmillan, pp 1\u20139. https:\/\/doi.org\/10.1057\/978-1-349-95121-5_1393-2"},{"issue":"1","key":"15_CR23","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1145\/1842890.1842893","volume":"44","author":"A Hanbury","year":"2010","unstructured":"Hanbury A, Zenz V, Berger H (2010) 1st international workshop on advances in patent information retrieval (AsPIRe\u201910). SIGIR Forum 44(1):19\u201322. http:\/\/doi.acm.org\/10.1145\/1842890.1842893","journal-title":"SIGIR Forum"},{"key":"15_CR24","unstructured":"Hunt D, Nguyen L, Rodgers M (2007) Patent searching: tools & techniques. Wiley, New York"},{"key":"15_CR25","doi-asserted-by":"crossref","unstructured":"Iwayama M, Fujii A, Kando N, Marukawa Y (2003) An empirical study on retrieval models for different document genres: patents and newspaper articles. In: Proceedings of the 26th international ACM SIGIR conference on research and development in information retrieval. ACM, New York, SIGIR \u201803, pp 251\u2013258","DOI":"10.1145\/860435.860482"},{"issue":"4","key":"15_CR26","doi-asserted-by":"publisher","first-page":"422","DOI":"10.1145\/582415.582418","volume":"20","author":"K J\u00e4rvelin","year":"2002","unstructured":"J\u00e4rvelin K, Kek\u00e4l\u00e4inen J (2002) Cumulated gain-based evaluation of IR techniques. ACM Trans Inf Syst 20(4):422\u2013446. http:\/\/doi.acm.org\/10.1145\/582415.582418","journal-title":"ACM Trans Inf Syst"},{"key":"15_CR27","doi-asserted-by":"crossref","unstructured":"Kamps J, Pehcevski J, Kazai G, Lalmas M, Robertson S (2008) INEX 2007 evaluation measures. In: Focused access to XML documents, 6th international workshop of the initiative for the evaluation of XML retrieval, INEX 2007, Dagstuhl Castle, Germany, December 17\u201319, 2007. Selected papers. Lecture notes in computer science, vol 4862. Springer, Berlin, pp 24\u201333","DOI":"10.1007\/978-3-540-85902-4_2"},{"issue":"1","key":"15_CR28","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1145\/373593.373621","volume":"34","author":"N Kando","year":"2000","unstructured":"Kando N, Leong MK (2000) Workshop on patent retrieval (SIGIR 2000 Workshop Report). SIGIR Forum 34(1):28\u201330","journal-title":"SIGIR Forum"},{"key":"15_CR29","unstructured":"Kumagai KI (2005) History of Japanese Industrial Property System. http:\/\/www.jpo.go.jp\/torikumi_e\/kokusai_e\/training\/textbook\/pdf\/History_of_Japanese_Industrial_Property_System(2005).pdf . Accessed Feb 2018"},{"key":"15_CR30","first-page":"357","volume-title":"Lecture Notes in Computer Science","author":"Aldo Lipani","year":"2017","unstructured":"Lipani A, Palotti J, Lupu M, Piroi F, Zuccon G, Hanbury A (2017) Fixed-cost pooling strategies based on IR evaluation measures. In: Advances in information retrieval - 39th European conference on IR research, ECIR 2017, Aberdeen, April 8\u201313, 2017, Proceedings, pp 357\u2013368. https:\/\/doi.org\/10.1007\/978-3-319-56608-5_28"},{"key":"15_CR31","unstructured":"Lopez P, Romary L (2009) Multiple retrieval models and regression models for prior art search. In: Borri F, Nardi A, Peters C, Ferro N (eds) (2009) CLEF 2009 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1175\/"},{"key":"15_CR32","unstructured":"Lopez P, Romary L (2010) Experiments with citation mining and key-term extraction for prior art search. In: Braschler M, Harman DK, Pianta E, Ferro N (eds) (2010) CLEF 2010 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1176\/"},{"key":"15_CR33","first-page":"1","volume":"7","author":"M Lupu","year":"2013","unstructured":"Lupu M, Hanbury A (2013) Patent retrieval. Found Trends IR 7:1\u201397","journal-title":"Found Trends IR"},{"issue":"2","key":"15_CR34","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1145\/1670564.1670576","volume":"43","author":"M Lupu","year":"2009","unstructured":"Lupu M, Huang J, Zhu J, Tait J (2009) TREC-CHEM: large scale chemical information retrieval evaluation at TREC. SIGIR Forum 43(2):63\u201370","journal-title":"SIGIR Forum"},{"key":"15_CR35","series-title":"The Information Retrieval Series","volume-title":"Current Challenges in Patent Information Retrieval","year":"2017","unstructured":"Lupu M, Mayer K, Kando N, Trippe A (2017) Current challenges in patent information retrieval, 2nd edn. Springer, Berlin. https:\/\/doi.org\/10.1007\/978-3-662-53817-3"},{"key":"15_CR36","doi-asserted-by":"crossref","unstructured":"Magdy W, Jones G (2010a) PRES: a score metric for evaluating recall-oriented information retrieval applications. In: Proceedings of the 33rd international ACM SIGIR conference on research and development in information retrieval, ACM, New York, SIGIR \u201810, pp 611\u2013618. http:\/\/doi.acm.org\/10.1145\/1835449.1835551","DOI":"10.1145\/1835449.1835551"},{"key":"15_CR37","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1007\/978-3-642-15998-5_10","volume-title":"Multilingual and Multimodal Information Access Evaluation","author":"Walid Magdy","year":"2010","unstructured":"Magdy W, Jones GJF (2010b) Examining the robustness of evaluation metrics for patent retrieval with incomplete relevance judgements. In: Braschler M, Harman DK, Pianta E, Ferro N (eds) (2010) CLEF 2010 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1176\/"},{"key":"15_CR38","unstructured":"Magdy W, Leveling J, Jones G (2009) DCU at CLEF-IP 2009: exploring standard IR techniques on patent retrieval. In: Borri F, Nardi A, Peters C, Ferro N (eds) (2009) CLEF 2009 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1175\/"},{"key":"15_CR39","unstructured":"Mahdabi P, Andersson L, Hanbury A, Crestani F (2011) Report on the CLEF-IP 2011 experiments: exploring patent summarization. In: Petras V, Forner P, Clough P, Ferro N (eds) (2011) CLEF 2011 Working Notes, CEUR Workshop Proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1177\/CLEF2011wn-CLEF-IP-MahdabiEt2011.pdf"},{"key":"15_CR40","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1080\/08109020210138979","volume":"20","author":"C May","year":"2010","unstructured":"May C (2010) The Venetian moment: new technologies, legal innovation and the institutional origins of intellectual property. Prometheus 20:159\u2013179. http:\/\/www.tandfonline.com\/doi\/full\/10.1080\/08109020210138979","journal-title":"Prometheus"},{"issue":"5","key":"15_CR41","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1021\/c160035a008","volume":"9","author":"P McDonnell","year":"1969","unstructured":"McDonnell P (1969) Technical information management in the U.S. patent office. J Chem Doc 9(5):220\u2013224","journal-title":"J Chem Doc"},{"issue":"5","key":"15_CR42","first-page":"953","volume":"92","author":"A Mossoff","year":"2007","unstructured":"Mossoff A (2007) Who cares What Thomas Jefferson thought about patents - reevaluating the patent privilege in historical context. Cornell Law Rev 92(5):953. https:\/\/scholarship.law.cornell.edu\/clr\/vol92\/iss5\/2","journal-title":"Cornell Law Rev"},{"key":"15_CR43","first-page":"216","volume-title":"Proceedings of the 6th international conference on information and knowledge management","author":"M Osborn","year":"1997","unstructured":"Osborn M, Strzalkowski T, Marinescu M (1997) Evaluating document retrieval in patent database: a preliminary report. In: Proceedings of the 6th international conference on information and knowledge management. ACM, New York, pp 216\u2013221. http:\/\/doi.acm.org\/10.1145\/266714.266899"},{"key":"15_CR44","unstructured":"PCT (1970) Patent Cooperation Treaty. http:\/\/www.wipo.int\/pct\/en\/treaty\/about.html . Accessed Aug 2015"},{"key":"15_CR45","unstructured":"Pfaller W (2013a) Bergrecht, Monopole, Privilegien. http:\/\/www.wolfgang-pfaller.de\/berg.htm . Accessed Feb 2018"},{"key":"15_CR46","unstructured":"Pfaller W (2013b) Schon die alten Griechen \u2026. http:\/\/www.wolfgang-pfaller.de\/sybaris.htm . Accessed Feb 2018"},{"key":"15_CR47","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1007\/978-3-642-19231-9_4","volume-title":"Current Challenges in Patent Information Retrieval","author":"Florina Piroi","year":"2011","unstructured":"Piroi F, Zenz V (2011) Evaluating information retrieval in the intellectual property domain: the CLEF-IP campaign. In: Lupu M, Mayer K, Tait J, Trippe AJ (eds) Current challenges in patent information retrieval, the information retrieval series, vol 29. Springer, Berlin, pp 87\u2013108. http:\/\/dx.doi.org\/10.1007\/978-3-642-19231-9_4"},{"key":"15_CR48","unstructured":"Piroi F, Lupu M, Hanbury A, Zenz V (2011) CLEF-IP 2011: retrieval in the intellectual property domain. In: Petras V, Forner P, Clough P, Ferro N (eds) (2011) CLEF 2011 Working Notes, CEUR Workshop Proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1177\/"},{"key":"15_CR49","unstructured":"Piroi F, Lupu M, Hanbury A, Sexton AP, Magdy W, Filippov IV (2012) CLEF-IP 2012: retrieval experiments in the intellectual property domain. In: Forner P, Karlgren J, Womser-Hacker C, Ferro N (eds) (2012) CLEF 2012 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1178\/"},{"key":"15_CR50","first-page":"232","volume-title":"Lecture Notes in Computer Science","author":"Florina Piroi","year":"2013","unstructured":"Piroi F, Lupu M, Hanbury A (2013) Overview of CLEF-IP 2013 Lab - information retrieval in the patent domain. In: Forner P, M\u00fcller H, Paredes R, Rosso P, Stein B (eds) Information access evaluation meets multilinguality, multimodality, and visualization. Proceedings of the fourth international conference of the CLEF initiative (CLEF 2013). Lecture Notes in Computer Science (LNCS), vol 8138. Springer, Heidelberg"},{"key":"15_CR51","unstructured":"Rich GS (1993) Are Letters Patent Grants of Monopoly? Western New England Law Review 15. https:\/\/core.ac.uk\/display\/76563380"},{"key":"15_CR52","series-title":"Text retrieval experiments 10th workshop of the cross-language evaluation forum, CLEF 2009, vol","first-page":"385","volume-title":"Multilingual information access evaluation I","author":"G Roda","year":"2010","unstructured":"Roda G, Tait J, Piroi F, Zenz V (2010) CLEF-IP 2009: retrieval experiments in the intellectual property domain. In: Peters C, Nunzio GD, Kurimo M, Mostefa D, Penas A, Roda G (eds) Multilingual information access evaluation I. Text retrieval experiments 10th workshop of the cross-language evaluation forum, CLEF 2009, vol 6241. Springer, Berlin, pp 385\u2013409"},{"issue":"3","key":"15_CR53","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1016\/S0172-2190(02)00019-4","volume":"24","author":"I Schneller","year":"2002","unstructured":"Schneller I (2002) Japanese File Index Classification and F-terms. World Patent Inf 24(3):197\u2013201","journal-title":"World Patent Inf"},{"issue":"3","key":"15_CR54","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1021\/ci60011a002","volume":"17","author":"H Skolnik","year":"1977","unstructured":"Skolnik H (1977) Historical aspects of patent systems. J Chem Inf Comput Sci 17(3):119\u2013121. https:\/\/pubs.acs.org\/doi\/abs\/10.1021\/ci60011a002","journal-title":"J Chem Inf Comput Sci"},{"key":"15_CR55","unstructured":"Spark-Jones K, Van Rijsbergen C (1975) Report on the need for and provision of an \u2018ideal\u2019 information retrieval test collection. 5266, Computer Laboratory, Univ. Cambridge. http:\/\/sigir.org\/files\/museum\/pub-14\/pub_14.pdf"},{"key":"15_CR56","unstructured":"Szarvas G, Herbert B, Ggurevych I (2009) Prior art search using international patent classification codes and all-claims-queries. In: Borri F, Nardi A, Peters C, Ferro N (eds) (2009) CLEF 2009 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1175\/"},{"key":"15_CR57","unstructured":"Tait J, Harris C, Lupu M (eds) (2010) PaIR \u201810: proceedings of the 3rd international workshop on patent information retrieval. ACM, New York"},{"key":"15_CR58","unstructured":"Teodoro D, Gobeill J, Pasche E, Vishnyakova D, Ruch P, Lovis C (2010) Automatic prior art searching and patent encoding at CLEF-IP\u201910. In: Braschler M, Harman DK, Pianta E, Ferro N (eds) (2010) CLEF 2010 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1176\/"},{"key":"15_CR59","unstructured":"Toucedo C, Losada D (2009) University of Santiago de Compostella at CLEF-IP\u201909. In: Borri F, Nardi A, Peters C, Ferro N (eds) (2009) CLEF 2009 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1175\/"},{"key":"15_CR60","unstructured":"Verberne S, D\u2019hondt E (2011) Patent classification experiments with the Linguistic Classification System LCS in CLEF-IP 2011. In: Petras V, Forner P, Clough P, Ferro N (eds) (2011) CLEF 2011 Working Notes, CEUR Workshop Proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1177\/"},{"key":"15_CR61","volume-title":"TREC: experiment and evaluation in information retrieval (digital libraries and electronic publishing)","author":"EM Voorhees","year":"2005","unstructured":"Voorhees EM, Harman DK (2005) TREC: experiment and evaluation in information retrieval (digital libraries and electronic publishing). The MIT Press, Cambridge"},{"key":"15_CR62","unstructured":"Wanagiri M, Adriani M (2010) Prior art retrieval using various patent document fields contents. In: Braschler M, Harman DK, Pianta E, Ferro N (eds) (2010) CLEF 2010 working notes. CEUR workshop proceedings (CEUR-WS.org), ISSN 1613-0073. http:\/\/ceur-ws.org\/Vol-1176\/"},{"key":"15_CR63","unstructured":"WIPO (2013) Handbook on industrial property information and documentation. Part 8: terms and abbreviations concerning industrial property information and documentation. http:\/\/www.wipo.int\/standards\/en\/part_08.html . Accessed Jan 2019"}],"container-title":["The Information Retrieval Series","Information Retrieval Evaluation in a Changing World"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-030-22948-1_15","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,9,25]],"date-time":"2022-09-25T14:18:55Z","timestamp":1664115535000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/978-3-030-22948-1_15"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019]]},"ISBN":["9783030229474","9783030229481"],"references-count":63,"URL":"https:\/\/doi.org\/10.1007\/978-3-030-22948-1_15","relation":{},"ISSN":["1387-5264"],"issn-type":[{"type":"print","value":"1387-5264"}],"subject":[],"published":{"date-parts":[[2019]]},"assertion":[{"value":"14 August 2019","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}}]}}