{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T12:51:23Z","timestamp":1775047883000,"version":"3.50.1"},"reference-count":58,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,3,11]],"date-time":"2023-03-11T00:00:00Z","timestamp":1678492800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,11]],"date-time":"2023-03-11T00:00:00Z","timestamp":1678492800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100012470","name":"CERN","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100012470","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Comput Softw Big Sci"],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The ATLAS EventIndex system comprises the catalogue of all events collected, processed or generated by the ATLAS experiment at the CERN LHC accelerator, and all associated software tools to collect, store and query this information. ATLAS records several billion particle interactions every year of operation, processes them for analysis and generates even larger simulated data samples; a global catalogue is needed to keep track of the location of each event record and be able to search and retrieve specific events for in-depth investigations. Each EventIndex record includes summary information on the event itself and the pointers to the files containing the full event. Most components of the EventIndex system are implemented using BigData free and open-source software. This paper describes the architectural choices and their evolution in time, as well as the past, current and foreseen future implementations of all EventIndex components.<\/jats:p>","DOI":"10.1007\/s41781-023-00096-8","type":"journal-article","created":{"date-parts":[[2023,3,26]],"date-time":"2023-03-26T21:49:57Z","timestamp":1679867397000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["The ATLAS EventIndex"],"prefix":"10.1007","volume":"7","author":[{"given":"Dario","family":"Barberis","sequence":"first","affiliation":[]},{"given":"Igor","family":"Alexandrov","sequence":"additional","affiliation":[]},{"given":"Evgeny","family":"Alexandrov","sequence":"additional","affiliation":[]},{"given":"Zbigniew","family":"Baranowski","sequence":"additional","affiliation":[]},{"given":"Luca","family":"Canali","sequence":"additional","affiliation":[]},{"given":"Elizaveta","family":"Cherepanova","sequence":"additional","affiliation":[]},{"given":"Gancho","family":"Dimitrov","sequence":"additional","affiliation":[]},{"given":"Andrea","family":"Favareto","sequence":"additional","affiliation":[]},{"given":"\u00c1lvaro","family":"Fern\u00e1ndez Casan\u00ed","sequence":"additional","affiliation":[]},{"given":"Elizabeth J.","family":"Gallas","sequence":"additional","affiliation":[]},{"given":"Carlos Garc\u00eda","family":"Montoro","sequence":"additional","affiliation":[]},{"given":"Santiago","family":"Gonz\u00e1lez de la Hoz","sequence":"additional","affiliation":[]},{"given":"Julius","family":"H\u0159ivn\u00e1\u010d","sequence":"additional","affiliation":[]},{"given":"Alexander","family":"Iakovlev","sequence":"additional","affiliation":[]},{"given":"Andrei","family":"Kazymov","sequence":"additional","affiliation":[]},{"given":"Mikhail","family":"Mineev","sequence":"additional","affiliation":[]},{"given":"Fedor","family":"Prokoshin","sequence":"additional","affiliation":[]},{"given":"Grigori","family":"Rybkin","sequence":"additional","affiliation":[]},{"given":"Jos\u00e9","family":"Salt","sequence":"additional","affiliation":[]},{"given":"Javier","family":"S\u00e1nchez","sequence":"additional","affiliation":[]},{"given":"Roman","family":"Sorokoletov","sequence":"additional","affiliation":[]},{"given":"Rainer","family":"T\u00f6bbicke","sequence":"additional","affiliation":[]},{"given":"Petya","family":"Vasileva","sequence":"additional","affiliation":[]},{"given":"Miguel","family":"Villaplana Perez","sequence":"additional","affiliation":[]},{"given":"Ruijun","family":"Yuan","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,3,11]]},"reference":[{"key":"96_CR1","doi-asserted-by":"publisher","unstructured":"Collaboration ATLAS (2008) The ATLAS experiment at the CERN Large Hadron Collider. JINST 3:S08003. https:\/\/doi.org\/10.1088\/1748-0221\/3\/08\/S08003","DOI":"10.1088\/1748-0221\/3\/08\/S08003"},{"key":"96_CR2","doi-asserted-by":"publisher","unstructured":"Evans L, Bryant P (2008) LHC machine. JINST 3:S08001. https:\/\/doi.org\/10.1088\/1748-0221\/3\/08\/S08001","DOI":"10.1088\/1748-0221\/3\/08\/S08001"},{"key":"96_CR3","doi-asserted-by":"publisher","unstructured":"Collaboration ATLAS (2020) Operation of the ATLAS trigger system in Run 2. JINST 15:P10004. https:\/\/doi.org\/10.1088\/1748-0221\/15\/10\/P10004","DOI":"10.1088\/1748-0221\/15\/10\/P10004"},{"key":"96_CR4","unstructured":"LHC Computing Grid: Technical Design Report. Document LCG-TDR-001, CERN-LHCC-2005-024. 20 June 2005. ISBN 978-92-9083-253-9"},{"key":"96_CR5","unstructured":"GUID: http:\/\/guid.one"},{"key":"96_CR6","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1007\/s41781-019-0026-3","volume":"3","author":"M Barisits","year":"2019","unstructured":"Barisits M, Beermann T, Berghaus F et al (2019) Rucio: scientific data management. Comput Softw Big Sci 3:11. https:\/\/doi.org\/10.1007\/s41781-019-0026-3","journal-title":"Comput Softw Big Sci"},{"key":"96_CR7","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/396\/5\/052053","volume":"396","author":"D Malon","year":"2012","unstructured":"Malon D et al (2012) An extensible infrastructure for querying and mining event-level metadata in ATLAS. J Phys Conf Ser 396:052053. https:\/\/doi.org\/10.1088\/1742-6596\/396\/5\/052053","journal-title":"J Phys Conf Ser"},{"key":"96_CR8","unstructured":"Oracle: https:\/\/www.oracle.com"},{"key":"96_CR9","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/513\/4\/042009","volume":"513","author":"D Barberis","year":"2014","unstructured":"Barberis D et al (2014) The future of event-level information repositories, indexing, and selection in ATLAS. J Phys Conf Ser 513:042009. https:\/\/doi.org\/10.1088\/1742-6596\/513\/4\/042009","journal-title":"J Phys Conf Ser"},{"key":"96_CR10","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/513\/4\/042002","volume":"513","author":"D Barberis","year":"2014","unstructured":"Barberis D et al (2014) The ATLAS Eventindex: an event catalogue for experiments collecting large amounts of data. J Phys Conf Ser 513:042002. https:\/\/doi.org\/10.1088\/1742-6596\/513\/4\/042002","journal-title":"J Phys Conf Ser"},{"key":"96_CR11","unstructured":"Hadoop: https:\/\/hadoop.apache.org"},{"key":"96_CR12","unstructured":"HBase: https:\/\/hbase.apache.org"},{"key":"96_CR13","unstructured":"ATLAS Event Displays: https:\/\/twiki.cern.ch\/twiki\/bin\/view\/AtlasPublic\/EventDisplayRun2Collisions"},{"key":"96_CR14","doi-asserted-by":"publisher","unstructured":"Alexandrov E et al.\u00a0(2021) Development of the ATLAS Event picking server. In:\u00a0Proc. 9th Int. Conf. \u201cDistributed Computing and Grid Technologies in Science and Education\u201d (GRID\u20192021), Dubna (Russia). https:\/\/doi.org\/10.54546\/MLIT.2021.35.43.001","DOI":"10.54546\/MLIT.2021.35.43.001"},{"key":"96_CR15","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/219\/7\/072011","volume":"219","author":"M Elsing","year":"2010","unstructured":"Elsing M et al (2010) The ATLAS Tier-0: overview and operational experience. J Phys Conf Ser 219:072011. https:\/\/doi.org\/10.1088\/1742-6596\/219\/7\/072011","journal-title":"J Phys Conf Ser"},{"key":"96_CR16","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/898\/6\/062001","volume":"898","author":"J Fulachier","year":"2017","unstructured":"Fulachier J et al (2017) ATLAS Metadata Interface (AMI), a generic metadata framework. J Phys Conf Ser 898:062001. https:\/\/doi.org\/10.1088\/1742-6596\/898\/6\/062001","journal-title":"J Phys Conf Ser"},{"key":"96_CR17","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/898\/5\/052002","volume":"898","author":"FH Barreiro Megino","year":"2017","unstructured":"Barreiro Megino FH et al (2017) PanDA for ATLAS distributed computing in the next decade. J Phys Conf Ser 898:052002. https:\/\/doi.org\/10.1088\/1742-6596\/898\/5\/052002","journal-title":"J Phys Conf Ser"},{"key":"96_CR18","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/513\/3\/032094","volume":"513","author":"GA Stewart","year":"2014","unstructured":"Stewart GA et al (2014) ATLAS job transforms: a data driven workflow engine. J Phys Conf Ser 513:032094. https:\/\/doi.org\/10.1088\/1742-6596\/513\/3\/032094","journal-title":"J Phys Conf Ser"},{"key":"96_CR19","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/762\/1\/012024","volume":"762","author":"GA Stewart","year":"2016","unstructured":"Stewart GA et al (2016) Multi-threaded software framework development for the ATLAS experiment. J Phys Conf Ser 762:012024. https:\/\/doi.org\/10.1088\/1742-6596\/762\/1\/012024","journal-title":"J Phys Conf Ser"},{"key":"96_CR20","unstructured":"Duellmann D (2003) The LCG POOL project: General overview and project structure. In: Proc. Computing in High Energy and Nuclear Physics (CHEP03), La Jolla, Ca, USA. https:\/\/arxiv.org\/abs\/physics\/0306129"},{"key":"96_CR21","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1016\/S0168-9002(97)00048-X","volume":"389","author":"R Brun","year":"1997","unstructured":"Brun R, Rademakers F (1997) ROOT\u2014an object oriented data analysis framework. Nucl Inst Meth Phys Res A 389:81\u201386. https:\/\/doi.org\/10.1016\/S0168-9002(97)00048-X","journal-title":"Nucl Inst Meth Phys Res A"},{"key":"96_CR22","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/898\/6\/062010","volume":"898","author":"A Fern\u00e1ndez Casan\u00ed","year":"2017","unstructured":"Fern\u00e1ndez Casan\u00ed A et al (2017) ATLAS EventIndex general dataflow and monitoring infrastructure. J Phys Conf Ser 898:062010. https:\/\/doi.org\/10.1088\/1742-6596\/898\/6\/062010","journal-title":"J Phys Conf Ser"},{"key":"96_CR23","unstructured":"ActiveMQ: http:\/\/activemq.apache.org"},{"key":"96_CR24","unstructured":"STOMP: https:\/\/stomp.github.io"},{"key":"96_CR25","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1007\/s10723-021-09580-0","volume":"19","author":"A Fern\u00e1ndez Casan\u00ed","year":"2021","unstructured":"Fern\u00e1ndez Casan\u00ed A et al (2021) A reliable large distributed object store based platform for collecting event metadata. J Grid Comp 19:39. https:\/\/doi.org\/10.1007\/s10723-021-09580-0","journal-title":"J Grid Comp"},{"key":"96_CR26","doi-asserted-by":"crossref","unstructured":"Mesnier M, Ganger GR, Riedel E (2003) IEEE Communications Magazine 41. 84-90 ISSN 0163-6804","DOI":"10.1109\/MCOM.2003.1222722"},{"key":"96_CR27","unstructured":"EOS: https:\/\/eos-docs.web.cern.ch"},{"key":"96_CR28","unstructured":"Xrdcp: https:\/\/xrootd.slac.stanford.edu"},{"key":"96_CR29","unstructured":"SQLite3: http:\/\/www.sqlite.org\/sqlite.html"},{"key":"96_CR30","unstructured":"Google Protocol Buffers (Google\u2019s Data Interchange Format): http:\/\/code.google.com\/apis\/protocolbuffers"},{"key":"96_CR31","unstructured":"Gzip: https:\/\/www.gzip.org"},{"key":"96_CR32","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/331\/7\/072034","volume":"331","author":"I Ueda","year":"2011","unstructured":"Ueda I et al (2011) ATLAS operations: experience and evolution in the data taking era. J Phys Conf Ser 331:072034. https:\/\/doi.org\/10.1088\/1742-6596\/331\/7\/072034","journal-title":"J Phys Conf Ser"},{"key":"96_CR33","unstructured":"PanDA Monitoring: https:\/\/bigpanda.cern.ch"},{"key":"96_CR34","unstructured":"Future pattern: https:\/\/docs.oracle.com\/javase\/7\/docs\/ api\/java\/util\/concurrent\/Future.html"},{"key":"96_CR35","unstructured":"Kudu: https:\/\/kudu.apache.org"},{"key":"96_CR36","unstructured":"Phoenix: https:\/\/phoenix.apache.org"},{"key":"96_CR37","unstructured":"Hadoop MapFile: https:\/\/hadoop.apache.org\/docs\/ r2.6.2\/api\/org\/apache\/hadoop\/io\/MapFile.html"},{"key":"96_CR38","unstructured":"Hadoop SequenceFile: https:\/\/hadoop.apache.org\/docs\/r2.6.2\/api\/org\/apache\/hadoop\/io\/SequenceFile.html"},{"key":"96_CR39","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/513\/4\/042020","volume":"513","author":"EJ Gallas","year":"2014","unstructured":"Gallas EJ et al (2014) Utility of collecting metadata to manage a large scale conditions database in ATLAS. J Phys Conf Ser 513:042020. https:\/\/doi.org\/10.1088\/1742-6596\/513\/4\/042020","journal-title":"J Phys Conf Ser"},{"key":"96_CR40","unstructured":"Mineev M, Prokoshin F and Yakovlev A (2018) Trigger information data flow for the ATLAS EventIndex. In: Proceedings of the VIII International Conference \u201cDistributed Computing and Grid-technologies in Science and Education\u201d (GRID 2018), Dubna (Russia), http:\/\/ceur-ws.org\/Vol-2267\/104-107-paper-18.pdf"},{"key":"96_CR41","unstructured":"CVMFS: https:\/\/cernvm.cern.ch\/fs"},{"key":"96_CR42","unstructured":"Tomcat: https:\/\/tomcat.apache.org"},{"key":"96_CR43","unstructured":"REST: https:\/\/restfulapi.net"},{"key":"96_CR44","unstructured":"Apache License 2.0: http:\/\/www.apache.org\/licenses\/LICENSE-2.0"},{"key":"96_CR45","unstructured":"Free Software Definition: https:\/\/www.gnu.org\/philosophy\/free-sw.html"},{"key":"96_CR46","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/898\/4\/042033","volume":"898","author":"EJ Gallas","year":"2017","unstructured":"Gallas EJ et al (2017) An Oracle-based event index for ATLAS. J Phys Conf Ser 898:042033. https:\/\/doi.org\/10.1088\/1742-6596\/898\/4\/042033","journal-title":"J Phys Conf Ser"},{"key":"96_CR47","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/664\/7\/072007","volume":"664","author":"J Catmore","year":"2015","unstructured":"Catmore J et al (2015) A new petabyte-scale data derivation framework for ATLAS. J Phys Conf Ser 664:072007. https:\/\/doi.org\/10.1088\/1742-6596\/664\/7\/072007","journal-title":"J Phys Conf Ser"},{"key":"96_CR48","unstructured":"Kibana: https:\/\/www.elastic.co\/products\/kibana"},{"key":"96_CR49","unstructured":"InfluxDB: https:\/\/www.influxdata.com"},{"key":"96_CR50","unstructured":"Grafana: https:\/\/grafana.com"},{"key":"96_CR51","unstructured":"Alexandrov E et al. (2018) BigData Tools for the Monitoring of the ATLAS EventIndex. In: Proc. VIII Int. Conf. \u201cDistributed Computing and Grid-technologies in Science and Education\u201d (GRID\u20192018), Dubna (Russia), http:\/\/ceur-ws.org\/Vol-2267\/91-94-paper-15.pdf"},{"key":"96_CR52","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/898\/6\/062020","volume":"898","author":"Z Baranowski","year":"2017","unstructured":"Baranowski Z et al (2017) A study of data representation in Hadoop to optimize data storage and search performance for the ATLAS EventIndex. J Phys Conf Ser 898:062020. https:\/\/doi.org\/10.1088\/1742-6596\/898\/6\/062020","journal-title":"J Phys Conf Ser"},{"key":"96_CR53","doi-asserted-by":"publisher","first-page":"04057","DOI":"10.1051\/epjconf\/201921404057","volume":"214","author":"Z Baranowski","year":"2019","unstructured":"Baranowski Z et al (2019) A prototype for the evolution of ATLAS EventIndex based on Apache Kudu storage. EPJ Web Conf 214:04057. https:\/\/doi.org\/10.1051\/epjconf\/201921404057","journal-title":"EPJ Web Conf"},{"key":"96_CR54","doi-asserted-by":"publisher","unstructured":"Cherepanova E et al. (2021) The ATLAS EventIndex using the HBase\/Phoenix storage solution. In: Proc. 9th Int. Conf. \u201cDistributed Computing and Grid Technologies in Science and Education\u201d (GRID\u20192021), Dubna (Russia). https:\/\/doi.org\/10.54546\/MLIT.2021.68.25.001","DOI":"10.54546\/MLIT.2021.68.25.001"},{"key":"96_CR55","unstructured":"A Python client library for accessing messaging servers using the STOMP protocol: https:\/\/github.com\/jasonrbriggs\/stomp.py"},{"key":"96_CR56","unstructured":"An Amazon Web Services (AWS) Software Development Kit (SDK) for Python: https:\/\/github.com\/boto\/boto3"},{"issue":"11","key":"96_CR57","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1145\/2934664","volume":"59","author":"M Zaharia","year":"2016","unstructured":"Zaharia M et al (2016) Apache Spark: a unified engine for big data processing. Commun ACM 59(11):56\u201365. https:\/\/doi.org\/10.1145\/2934664","journal-title":"Commun ACM"},{"key":"96_CR58","unstructured":"Gremlin: https:\/\/tinkerpop.apache.org\/gremlin.html"}],"container-title":["Computing and Software for Big Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41781-023-00096-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s41781-023-00096-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41781-023-00096-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,26]],"date-time":"2023-03-26T21:50:06Z","timestamp":1679867406000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s41781-023-00096-8"}},"subtitle":["A BigData Catalogue for All ATLAS Experiment Events"],"short-title":[],"issued":{"date-parts":[[2023,3,11]]},"references-count":58,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["96"],"URL":"https:\/\/doi.org\/10.1007\/s41781-023-00096-8","relation":{},"ISSN":["2510-2036","2510-2044"],"issn-type":[{"value":"2510-2036","type":"print"},{"value":"2510-2044","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,11]]},"assertion":[{"value":"15 November 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 February 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 March 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interests"}}],"article-number":"2"}}