{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T20:21:33Z","timestamp":1774729293734,"version":"3.50.1"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,11,11]],"date-time":"2019-11-11T00:00:00Z","timestamp":1573430400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,11,11]],"date-time":"2019-11-11T00:00:00Z","timestamp":1573430400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Big Data"],"published-print":{"date-parts":[[2019,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n              <jats:p>In this paper we are proposing an adaptive and real-time approach to resolve real-time financial data integration latency problems and semantic heterogeneity. Due to constraints that we have faced in some projects that requires real-time massive financial data integration and analysis, we decided to follow a new approach by combining a hybrid financial ontology, resilient distributed datasets and real-time discretized stream. We create a real-time data integration pipeline to avoid all problems of classic Extract-Transform-Load tools, which are data processing latency, functional miscomprehensions and metadata heterogeneity. This approach is considered as contribution to enhance reporting quality and availability in short time frames, the reason of the use of Apache Spark. We studied Extract-Transform-Load (ETL) concepts, data warehousing fundamentals, big data processing technics and oriented containers clustering architecture, in order to replace the classic data integration and analysis process by our new concept resilient distributed DataStream for online analytical process (RDD4OLAP) cubes which are consumed by using Spark SQL or Spark Core basics.<\/jats:p>","DOI":"10.1186\/s40537-019-0260-x","type":"journal-article","created":{"date-parts":[[2019,11,11]],"date-time":"2019-11-11T08:03:08Z","timestamp":1573459388000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":30,"title":["An adaptive and real-time based architecture for financial data integration"],"prefix":"10.1186","volume":"6","author":[{"given":"Noussair","family":"Fikri","sequence":"first","affiliation":[]},{"given":"Mohamed","family":"Rida","sequence":"additional","affiliation":[]},{"given":"Noureddine","family":"Abghour","sequence":"additional","affiliation":[]},{"given":"Khalid","family":"Moussaid","sequence":"additional","affiliation":[]},{"given":"Amina","family":"El Omri","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2019,11,11]]},"reference":[{"issue":"8","key":"260_CR1","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1145\/1978542.1978562","volume":"54","author":"S Chaudhuri","year":"2011","unstructured":"Chaudhuri S, Dayal U, Narasayya V. An overview of business intelligence technology. Commun ACM. 2011;54(8):88\u201398.","journal-title":"Commun ACM"},{"issue":"1","key":"260_CR2","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1145\/248603.248616","volume":"26","author":"S Chaudhuri","year":"1997","unstructured":"Chaudhuri S, Dayal U. An overview of data warehousing and OLAP technology. ACM Sigmod Rec. 1997;26(1):65\u201374.","journal-title":"ACM Sigmod Rec"},{"key":"260_CR3","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-99984-5","volume-title":"Financial statements: analysis and reporting","author":"FI Lessambo","year":"2018","unstructured":"Lessambo FI. Financial statements: analysis and reporting. London: Palgrave Macmillan; 2018."},{"key":"260_CR4","doi-asserted-by":"crossref","unstructured":"Wibowo A. Problems and available solutions on the stage of extract, transform, and loading in near real-time data warehousing (a literature study). In: 2015 international seminar on intelligent technology and its applications (ISITIA); 2015. p. 345\u201350.","DOI":"10.1109\/ISITIA.2015.7220004"},{"key":"260_CR5","doi-asserted-by":"crossref","unstructured":"Su J, Tang Y. Business intelligence revisited. In: 2017 fifth international conference on advanced cloud and big data (CBD); 2017. p. 1\u20136.","DOI":"10.1109\/CBD.2017.9"},{"key":"260_CR6","doi-asserted-by":"crossref","unstructured":"Obal\u0131 M, Dursun B, Erdem Z, G\u00f6r\u00fcr AK. A real time data warehouse approach for data processing. In: 2013 21st signal processing and communications applications conference (SIU); 2013. p. 1\u20134.","DOI":"10.1109\/SIU.2013.6531245"},{"key":"260_CR7","unstructured":"Zaharia M, et al. Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: pr\u00e9sent\u00e9 \u00e0 presented as part of the 9th {USENIX} symposium on networked systems design and implementation ({NSDI} 12); 2012. p. 15\u201328."},{"issue":"3","key":"260_CR8","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1007\/s41060-016-0027-9","volume":"1","author":"S Salloum","year":"2016","unstructured":"Salloum S, Dautov R, Chen X, Peng PX, Huang JZ. Big data analytics on Apache Spark. Int J Data Sci Anal. 2016;1(3):145\u201364.","journal-title":"Int J Data Sci Anal"},{"issue":"1","key":"260_CR9","first-page":"1765","volume":"22","author":"M Chae","year":"2017","unstructured":"Chae M, Lee H, Lee K. A performance comparison of linux containers and virtual machines using Docker and KVM. Clust Comput. 2017;22(1):1765\u201375.","journal-title":"Clust Comput"},{"key":"260_CR10","doi-asserted-by":"crossref","unstructured":"Joy AM. Performance comparison between Linux containers and virtual machines. In: 2015 international conference on advances in computer engineering and applications; 2015. p. 342\u20136.","DOI":"10.1109\/ICACEA.2015.7164727"},{"key":"260_CR11","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4842-3024-4","volume-title":"Practical LXC and LXD: linux containers for virtualization and orchestration","author":"S Senthil Kumaran","year":"2017","unstructured":"Senthil Kumaran S. Practical LXC and LXD: linux containers for virtualization and orchestration. Berkele: Apress; 2017."},{"key":"260_CR12","volume-title":"Olap solutions: building multidimensional information systems","author":"E Thomsen","year":"2002","unstructured":"Thomsen E. Olap solutions: building multidimensional information systems. 2nd ed. New York: Wiley; 2002.","edition":"2"},{"key":"260_CR13","doi-asserted-by":"crossref","unstructured":"Gulla JA, Brasethvik T. A hybrid approach to ontology relationship learning. In: International conference on application of natural language and information systems; 2008. p. 79\u201390.","DOI":"10.1007\/978-3-540-69858-6_9"},{"key":"260_CR14","doi-asserted-by":"publisher","first-page":"012116","DOI":"10.1088\/1742-6596\/803\/1\/012116","volume":"803","author":"GG Petrova","year":"2017","unstructured":"Petrova GG, Tuzovsky AF, Aksenova NV. Application of the financial industry business ontology (FIBO) for development of a financial organization ontology. J Phys Conf Ser. 2017;803:012116.","journal-title":"J Phys Conf Ser"},{"issue":"1","key":"260_CR15","first-page":"2527","volume":"22","author":"CN Nguyen","year":"2018","unstructured":"Nguyen CN, Lee J, Hwang S, Kim JS. On the role of message broker middleware for many-task computing on a big-data platform. Clust Comput. 2018;22(1):2527\u201340.","journal-title":"Clust Comput"},{"key":"260_CR16","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-13-0550-4","volume-title":"Big data processing using spark in cloud","author":"M Mittal","year":"2019","unstructured":"Mittal M, Balas VE, Goyal LM, Kumar R. Big data processing using spark in cloud. Springer Singapore: Singapore; 2019."},{"key":"260_CR17","doi-asserted-by":"publisher","first-page":"2013","DOI":"10.1007\/978-3-642-27358-2","volume-title":"Business intelligence: second European summer school, eBISS, Brussels, Belgium, July 15\u201321, 2012, tutorial lectures","author":"MA Aufaure","year":"2012","unstructured":"Aufaure MA, Zim\u00e1nyi E. Business intelligence: second European summer school, eBISS, Brussels, Belgium, July 15\u201321, 2012, tutorial lectures. Berlin: Springer; 2012. p. 2013."},{"issue":"2","key":"260_CR18","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1007\/s10997-016-9344-6","volume":"20","author":"M Page","year":"2016","unstructured":"Page M, Spira LF. Special issue on \u201cbusiness models, financial reporting and corporate governance\u201d. J Manag Gov. 2016;20(2):209\u201311.","journal-title":"J Manag Gov"},{"issue":"9","key":"260_CR19","first-page":"9","volume":"2","author":"GS Reddy","year":"2010","unstructured":"Reddy GS, Srinivasu R, Rao MPC, Rikkula SR. Data warehousing, data mining OLAP and OLTP technologies are essential elements to support decision-making process in industries. J Comput Sci Eng. 2010;2(9):9.","journal-title":"J Comput Sci Eng"},{"key":"260_CR20","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4842-3579-9","volume-title":"Beginning apache spark 2: with resilient distributed datasets, spark SQL, structured streaming and spark machine learning library","author":"H Luu","year":"2018","unstructured":"Luu H. Beginning apache spark 2: with resilient distributed datasets, spark SQL, structured streaming and spark machine learning library. New York: Apress; 2018."},{"key":"260_CR21","unstructured":"Jun L, ChaoJu H, HeJin Y. Application of Web services on the real-time data warehouse technology. In: 2010 international conference on advances in energy engineering. 2010. p. 335\u20138."},{"key":"260_CR22","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-28225-1","volume-title":"IFRS in a global world: international and critical perspectives on accounting","author":"D Bensadon","year":"2016","unstructured":"Bensadon D, Praquin N. IFRS in a global world: international and critical perspectives on accounting. New York: Springer International Publishing; 2016."},{"key":"260_CR23","first-page":"389","volume-title":"Proceedings of 2013 4th International Asia Conference on Industrial Engineering and Management Innovation (IEMI2013)","author":"Yu-ju Li","year":"2013","unstructured":"Li Y, Li R. Research on financial statements system based on enterprise resource. In: Proceedings of 2013 4th international Asia conference on industrial engineering and management innovation (IEMI2013); 2014. p. 389\u201399."},{"issue":"2","key":"260_CR24","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1057\/jdg.2012.20","volume":"11","author":"J Cherny","year":"2014","unstructured":"Cherny J. Financial statements are messages that need a context to be better understood. Int J Discl Gov. 2014;11(2):161\u201376.","journal-title":"Int J Discl Gov"},{"key":"260_CR25","unstructured":"Real-time financial reporting: the need for speed July. https:\/\/www.google.co.ma\/search?source=hp&ei=iXZpXJivL4OmaLCVoNAE&q=REAL-TIME+FINANCIAL+REPORTING%3A+THE+NEED+FOR+SPEED+July%2C&btnK=Google+Search&oq=REAL-TIME+FINANCIAL+REPORTING%3A+THE+NEED+FOR+SPEED+July%2C&gs_l=psy-ab.3\u20261771.1771..2150\u20260.0..0.601.601.5-1\u2026\u20260\u2026.2j1..gws-wiz\u2026..0.yjmCb7-G2nE. Consult\u00e9 le: 17-f\u00e9vr-2019."},{"key":"260_CR26","doi-asserted-by":"crossref","unstructured":"de Heij D, Troyanovsky A, Yang C, Scharff MZ, Schouten K, Frasincar F. An ontology-enhanced hybrid approach to aspect-based sentiment analysis. In: Web information systems engineering\u2014WISE 2017; 2017. p. 338\u201345.","DOI":"10.1007\/978-3-319-68786-5_27"},{"key":"260_CR27","unstructured":"An extended hybrid ontology approach to data integration\u2014IEEE conference publication. https:\/\/ieeexplore.ieee.org\/document\/5304734. Consult\u00e9 le: 24-f\u00e9vr-2019."},{"key":"260_CR28","doi-asserted-by":"crossref","unstructured":"Li X, Mao Y. Real-time data ETL framework for big real-time data analysis. In: 2015 IEEE international conference on information and automation, Lijiang, China; 2015. p. 1289\u201394.","DOI":"10.1109\/ICInfA.2015.7279485"},{"key":"260_CR29","doi-asserted-by":"crossref","unstructured":"Mohammed Muddasir N, Raghuveer K. Study of methods to achieve near real time ETL. In: 2017 international conference on current trends in computer, electrical, electronics and communication (CTCEEC), Mysore, India; 2017. p. 436\u201341.","DOI":"10.1109\/CTCEEC.2017.8455002"}],"container-title":["Journal of Big Data"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-019-0260-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s40537-019-0260-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-019-0260-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,10]],"date-time":"2020-11-10T00:12:53Z","timestamp":1604967173000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-019-0260-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,11]]},"references-count":29,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,12]]}},"alternative-id":["260"],"URL":"https:\/\/doi.org\/10.1186\/s40537-019-0260-x","relation":{},"ISSN":["2196-1115"],"issn-type":[{"value":"2196-1115","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,11]]},"assertion":[{"value":"24 July 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 October 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 November 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare that they have no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"97"}}