{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,17]],"date-time":"2025-05-17T04:04:24Z","timestamp":1747454664647,"version":"3.40.5"},"reference-count":0,"publisher":"IOS Press","isbn-type":[{"value":"9781643685960","type":"electronic"}],"license":[{"start":{"date-parts":[[2025,5,15]],"date-time":"2025-05-15T00:00:00Z","timestamp":1747267200000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,5,15]]},"abstract":"<jats:p>This case study discusses the effectiveness of implementing a real-time automated monitoring architecture using the ELK Stack (Elasticsearch, Logstash and Kibana) to ensure data ingestion quality within the Medical Data Integration Center (MeDIC) at the University Hospital Cologne. By streamlining the ETL (Extract, Transform, and Load) log analysis process, this system minimizes the need for manual effort and brings increased efficiency and precision in analyzing data quality issues in real-time, detecting errors and potential problems, including the ability to uncover new errors. Over a six-month period, the implemented dashboard was able to process the ingestion logs of millions of files to provide valuable insights for the stakeholders in the decision-making process.<\/jats:p>","DOI":"10.3233\/shti250300","type":"book-chapter","created":{"date-parts":[[2025,5,16]],"date-time":"2025-05-16T08:53:16Z","timestamp":1747385596000},"source":"Crossref","is-referenced-by-count":0,"title":["Real-Time Automated Monitoring Architecture for Ensuring Quality ETL Processes in Healthcare Data: A Case Study from Cologne University Hospital\u2019s Medical Data Integration Center"],"prefix":"10.3233","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5506-4875","authenticated-orcid":false,"given":"Md. Mostafa","family":"Kamal","sequence":"first","affiliation":[{"name":"Institute for Biomedical Informatics, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne, Germany"},{"name":"Medical Data Integration Center (MeDIC), University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3430-5123","authenticated-orcid":false,"given":"Ekaterina","family":"Kutafina","sequence":"additional","affiliation":[{"name":"Institute for Biomedical Informatics, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7611-3501","authenticated-orcid":false,"given":"Oya","family":"Beyan","sequence":"additional","affiliation":[{"name":"Institute for Biomedical Informatics, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne, Germany"},{"name":"Medical Data Integration Center (MeDIC), University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne, Germany"},{"name":"Department of Data Science and Artificial Intelligence, Fraunhofer FIT, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"7437","container-title":["Studies in Health Technology and Informatics","Intelligent Health Systems \u2013 From Technology to Data and Knowledge"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/SHTI250300","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,16]],"date-time":"2025-05-16T08:53:16Z","timestamp":1747385596000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/SHTI250300"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,15]]},"ISBN":["9781643685960"],"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/shti250300","relation":{},"ISSN":["0926-9630","1879-8365"],"issn-type":[{"value":"0926-9630","type":"print"},{"value":"1879-8365","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,15]]}}}