{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T11:01:27Z","timestamp":1740135687095,"version":"3.37.3"},"reference-count":20,"publisher":"Cambridge University Press (CUP)","issue":"5","license":[{"start":{"date-parts":[[2017,5,9]],"date-time":"2017-05-09T00:00:00Z","timestamp":1494288000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Nat. Lang. Eng."],"published-print":{"date-parts":[[2017,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Computational power needs have greatly increased during the last years, and this is also the case in the Natural Language Processing (NLP) area, where thousands of documents must be processed, i.e., linguistically analyzed, in a reasonable time frame. These computing needs have implied a radical change in the computing architectures and big-scale text processing techniques used in NLP. In this paper, we present a scalable architecture for distributed language processing. The architecture uses Storm to combine diverse NLP modules into a processing chain, which carries out the linguistic analysis of documents. Scalability requires designing solutions that are able to run distributed programs in parallel and across large machine clusters. Using the architecture presented here, it is possible to integrate a set of third-party NLP modules into a unique processing chain which can be deployed onto a distributed environment, i.e., a cluster of machines, so allowing the language-processing modules run in parallel. No restrictions are placed a priori on the NLP modules apart of being able to consume and produce linguistic annotations following a given format. We show the feasibility of our approach by integrating two linguistic processing chains for English and Spanish. Moreover, we provide several scripts that allow building from scratch a whole distributed architecture that can be then easily installed and deployed onto a cluster of machines. The scripts and the NLP modules used in the paper are publicly available and distributed under free licenses. In the paper, we also describe a series of experiments carried out in the context of the NewsReader project with the goal of testing how the system behaves in different scenarios.<\/jats:p>","DOI":"10.1017\/s1351324917000092","type":"journal-article","created":{"date-parts":[[2017,5,9]],"date-time":"2017-05-09T04:52:04Z","timestamp":1494305524000},"page":"709-731","source":"Crossref","is-referenced-by-count":2,"title":["A scalable architecture for data-intensive natural language processing"],"prefix":"10.1017","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0469-5076","authenticated-orcid":false,"given":"ZUHAITZ","family":"BELOKI","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"XABIER","family":"ARTOLA","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"AITOR","family":"SOROA","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"56","published-online":{"date-parts":[[2017,5,9]]},"reference":[{"key":"S1351324917000092_ref020","unstructured":"Wu H. , Fei Z. , Dai A. , Sammons M. , Roth D. , and Mayhew S. D. 2014. Illinoiscloudnlp: text analytics services in the cloud. In Proceedings of International Conference on Language Resources and Evaluation (LREC), pp. 14\u201321."},{"key":"S1351324917000092_ref004","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2016.05.003"},{"volume-title":"Proceedings of the Language Resources and Evaluation Conference (LREC '12)","year":"2012","author":"Padr\u00f3","key":"S1351324917000092_ref016"},{"key":"S1351324917000092_ref002","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2014.11.007"},{"key":"S1351324917000092_ref008","doi-asserted-by":"publisher","DOI":"10.1007\/11816508_8"},{"key":"S1351324917000092_ref006","doi-asserted-by":"publisher","DOI":"10.1023\/A:1014348124664"},{"key":"S1351324917000092_ref015","first-page":"17","article-title":"An\u00e1lisis morfosint\u00e1ctico y clasificaci\u00f3n de entidades nombradas en un entorno Big Data","volume":"53","author":"Otero","year":"2014","journal-title":"Procesamiento del Lenguaje Natural"},{"key":"S1351324917000092_ref018","first-page":"1636","volume-title":"Proceedings of the 15th IEEE International Conference on Data Mining Workshop (ICDMW '15)","author":"Padr\u00f3","year":"2015"},{"volume-title":"Proceedings of the 9th Language Resources and Evaluation Conference (LREC2014)","year":"2014","author":"Agerri","key":"S1351324917000092_ref003"},{"key":"S1351324917000092_ref009","doi-asserted-by":"publisher","DOI":"10.1147\/JRD.2012.2188761"},{"volume-title":"CIDR 2003 \u2013 First Biennial Conference on Innovative Data Systems Research","year":"2003","author":"Cherniack","key":"S1351324917000092_ref005"},{"key":"S1351324917000092_ref010","first-page":"271","article-title":"Cocytus: parallel NLP over disparate data","volume":"49","author":"Evans","year":"2008","journal-title":"TAL"},{"key":"S1351324917000092_ref001","unstructured":"Agerri R. , Aldabe I. , Beloki Z. , Laparra E. , Rigau G. , Soroa A. , van Erp M. , Fokkens A. , Ilievski F. , Izquierdo R. , Morante R. , van Son C. , Vossen P. , and Minard A.-L. 2016. Event detection, version 3. NewsReader Deliverable 4.2.3."},{"key":"S1351324917000092_ref017","first-page":"1636","volume-title":"Proceedings of the IEEE International Conference on Data Mining Workshop (ICDMW)","author":"Padr\u00f3","year":"2015"},{"key":"S1351324917000092_ref019","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2012.0071"},{"key":"S1351324917000092_ref012","unstructured":"Fokkens A. , Soroa A. , Beloki Z. , Ockeloen N. , Rigau G. , van Hage W. R. , and Vossen P. 2014. NAF and GAF: linking linguistic annotations. In Proceedings of 10th Joint ACL\/ISO Workshop on Interoperable Semantic Annotation (ISA-10)."},{"volume-title":"Proceedings of the 21st International Conference on Distributed Multimedia Systems DMS '15","year":"2015","author":"Nesi","key":"S1351324917000092_ref014"},{"key":"S1351324917000092_ref007","doi-asserted-by":"publisher","DOI":"10.1145\/1327452.1327492"},{"key":"S1351324917000092_ref011","unstructured":"Exner P. , and Nugues P. 2014. KOSHIK: a large-scale distributed computing framework for NLP. In Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods, pp. 463\u201370."},{"key":"S1351324917000092_ref013","doi-asserted-by":"crossref","unstructured":"Ide N. , Romary L. , and de La Clergerie \u00c9. V. 2003. International standard for a linguistic annotation framework. In Proceedings of the HLT-NAACL 2003 Workshop on Software Engineering and Architecture of Language Technology Systems (SEALTS). Association for Computational Linguistics.","DOI":"10.3115\/1119226.1119230"}],"container-title":["Natural Language Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1351324917000092","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,4,16]],"date-time":"2019-04-16T20:44:38Z","timestamp":1555447478000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1351324917000092\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,5,9]]},"references-count":20,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2017,9]]}},"alternative-id":["S1351324917000092"],"URL":"https:\/\/doi.org\/10.1017\/s1351324917000092","relation":{},"ISSN":["1351-3249","1469-8110"],"issn-type":[{"type":"print","value":"1351-3249"},{"type":"electronic","value":"1469-8110"}],"subject":[],"published":{"date-parts":[[2017,5,9]]}}}