{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,22]],"date-time":"2026-03-22T06:24:51Z","timestamp":1774160691397,"version":"3.50.1"},"reference-count":53,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,4,6]],"date-time":"2020-04-06T00:00:00Z","timestamp":1586131200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,4,6]],"date-time":"2020-04-06T00:00:00Z","timestamp":1586131200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Big Data"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background<\/jats:title><jats:p>Augmented reality, computer vision and other (e.g. network functions, Internet-of-Things (IoT)) use cases can be realised in edge computing environments with machine learning (ML) techniques. For realisation of the use cases, it has to be understood how data is collected, stored, processed, analysed, and visualised in big data systems. In order to provide services with low latency for end users, often utilisation of ML techniques has to be optimized. Also, software\/service developers have to understand, how to develop and deploy ML models in edge computing environments. Therefore, architecture design of big data systems to edge computing environments may be challenging.<\/jats:p><\/jats:sec><jats:sec><jats:title>Findings<\/jats:title><jats:p>The contribution of this paper is reference architecture (RA) design of a big data system utilising ML techniques in edge computing environments. An earlier version of the RA has been extended based on 16 realised implementation architectures, which have been developed to edge\/distributed computing environments. Also, deployment of architectural elements in different environments is described. Finally, a system view is provided of the software engineering aspects of ML model development and deployment.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>The presented RA may facilitate concrete architecture design of use cases in edge computing environments. The value of RAs is reduction of development and maintenance costs of systems, reduction of risks, and facilitation of communication between different stakeholders.<\/jats:p><\/jats:sec>","DOI":"10.1186\/s40537-020-00303-y","type":"journal-article","created":{"date-parts":[[2020,4,6]],"date-time":"2020-04-06T16:03:57Z","timestamp":1586189037000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Extending reference architecture of big data systems towards machine learning in edge computing environments"],"prefix":"10.1186","volume":"7","author":[{"given":"P.","family":"P\u00e4\u00e4kk\u00f6nen","sequence":"first","affiliation":[]},{"given":"D.","family":"Pakkala","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,4,6]]},"reference":[{"key":"303_CR1","doi-asserted-by":"crossref","unstructured":"Chen GJ et al. Realtime data processing at facebook. In: ACMSIGMOD international conference on management of data, San Francisco, CA, USA, 16 June\u201301 July, 2016.","DOI":"10.1145\/2882903.2904441"},{"key":"303_CR2","doi-asserted-by":"publisher","first-page":"1634","DOI":"10.14778\/3137765.3137770","volume":"12","author":"SA Noghabi","year":"2017","unstructured":"Noghabi SA, et al. Samza: stateful scalable stream processing at LinkedIn. VLDB Endowment. 2017;12:1634\u201345.","journal-title":"VLDB Endowment."},{"key":"303_CR3","unstructured":"Fu M et al. Twitter heron: towards extensible streaming engines. In: IEEE 33rd international conference on data engineering, San Diego, CA, USA, 19\u201322 April, 2017, p. 1167\u201372."},{"key":"303_CR4","doi-asserted-by":"publisher","first-page":"166","DOI":"10.1016\/j.bdr.2015.01.001","volume":"4","author":"P P\u00e4\u00e4kk\u00f6nen","year":"2015","unstructured":"P\u00e4\u00e4kk\u00f6nen P, Pakkala D. Reference architecture and classification of technologies, products and services for big data systems. Big Data Res. 2015;4:166\u201386.","journal-title":"Big Data Res"},{"key":"303_CR5","unstructured":"Sena B, Allian AP, Nakagawa EY. Characterizing big data software architectures: a systematic mapping study. In: The 11th Brazilian symposium on software components, architectures and reuse (SBCARS), Fortaleza, Brazil, 18\u201319 September, 2017."},{"key":"303_CR6","volume-title":"Decision-support for selecting big data reference architectures. Business Information Systems (BIS 2019), lecture notes in business information processing","author":"M Volk","year":"2019","unstructured":"Volk M, Bosse S, Bischoff D, Turowski K. Decision-support for selecting big data reference architectures. Business Information Systems (BIS 2019), lecture notes in business information processing, vol. 353. Cham: Springer; 2019."},{"key":"303_CR7","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1109\/MWC.2017.1600304WC","volume":"24","author":"R Li","year":"2017","unstructured":"Li R, Zhifeng Z, Xuan Z, Guoru D, Yan C, Zhongyao W, Honggang Z. Intelligent 5G: When Cellular Networks Meet Artificial Intelligence. IEEE Wirel Commun. 2017;24:175\u201383.","journal-title":"IEEE Wirel Commun"},{"key":"303_CR8","doi-asserted-by":"publisher","first-page":"1655","DOI":"10.1109\/JPROC.2019.2921977","volume":"107","author":"J Chen","year":"2019","unstructured":"Chen J, Ran X. Deep learning with edge computing: a review. Proc IEEE. 2019;107:1655\u201374.","journal-title":"Proc IEEE"},{"key":"303_CR9","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1016\/j.infsof.2011.11.009","volume":"54","author":"A Angelov","year":"2012","unstructured":"Angelov A, Grefen P, Greefhorst D. A framework for analysis and design of software RAs. Inf Softw Technol. 2012;54:417\u201331.","journal-title":"Inf Softw Technol"},{"key":"303_CR10","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1016\/j.infsof.2017.03.011","volume":"88","author":"S Mart\u00ednez-Fern\u00e1ndez","year":"2017","unstructured":"Mart\u00ednez-Fern\u00e1ndez S, Ayala CP, Franch X, Marques HM. Benefits and drawbacks of software reference architectures: a case study. Inform Softw Tech. 2017;88:37\u201352.","journal-title":"Inform Softw Tech."},{"key":"303_CR11","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/j.scico.2018.03.007","volume":"163","author":"A Martini","year":"2018","unstructured":"Martini A, Besker T, Bosch J. Technical Debt tracking: current state of practice A survey and multiple case study in 15 large organizations. Sci Comput Program. 2018;163:42\u201361.","journal-title":"Sci Comput Program"},{"key":"303_CR12","doi-asserted-by":"crossref","unstructured":"Sang GM, Xu L, de Vrieze P. A reference architecture for big data systems. In: The 10th international conference on software, knowledge, information management & applications (SKIMA), Chengdu, China, 15\u201317 December, 2016, p. 370\u20135.","DOI":"10.1109\/SKIMA.2016.7916249"},{"key":"303_CR13","doi-asserted-by":"crossref","unstructured":"Sang GM, Xu L, de Vrieze P. Simplifying big data analytics systems with a reference architecture. In: The 18th IFIP WG 5.5 working conference on virtual enterprises, Vicenza, Italy, 18\u201320 September, 2017, p. 242\u20139.","DOI":"10.1007\/978-3-319-65151-4_23"},{"key":"303_CR14","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1016\/j.future.2014.10.016","volume":"47","author":"MA Mart\u00ednez-Prieto","year":"2015","unstructured":"Mart\u00ednez-Prieto MA, Cuesta CE, Arias M, Fern\u00e1ndez JD. The Solid architecture for real-time management of big semantic data. Future Gener Comp Sy. 2015;47:62\u201379.","journal-title":"Future Gener Comp Sy."},{"key":"303_CR15","volume-title":"Big data Principles and best practises of scalable real-time data systems","author":"N Marz","year":"2015","unstructured":"Marz N, Warren J. Big data Principles and best practises of scalable real-time data systems. New York: Manning Publications Co.; 2015."},{"key":"303_CR16","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1016\/j.infsof.2017.06.001","volume":"90","author":"S Nadal","year":"2017","unstructured":"Nadal S, Herrero V, Romero O, Abell\u00f3 A, Franch X, Vansummeren S, Valerio D. A software reference architecture for semantic-aware big data systems. Inform Software TecH. 2017;90:75\u201392.","journal-title":"Inform Software TecH."},{"key":"303_CR17","doi-asserted-by":"publisher","first-page":"96604","DOI":"10.1109\/ACCESS.2019.2929330","volume":"7","author":"J Moreno","year":"2019","unstructured":"Moreno J, Fernandez EB, Serrano MA, Fernandez M. Secure development of big data ecosystems. IEEE Access. 2019;7:96604\u201319.","journal-title":"IEEE Access"},{"key":"303_CR18","doi-asserted-by":"crossref","unstructured":"Roy B, Mondal AK, Roy CK, Schneider KA, Wazed K. Towards a reference architecture for Cloud-based plant genotyping and phenotyping analysis frameworks. In: IEEE international conference on software architecture, Gothenburg, Sweden, 03\u201307 April, 2017, p. 41\u201350.","DOI":"10.1109\/ICSA.2017.42"},{"key":"303_CR19","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1016\/j.ijinfomgt.2018.12.003","volume":"46","author":"M Lnenicka","year":"2019","unstructured":"Lnenicka M, Komarkova J. Developing a government enterprise architecture framework to support the requirements of big and open linked data with the use of cloud computing. Int J Inform Manage. 2019;46:124\u201341.","journal-title":"Int J Inform Manage."},{"key":"303_CR20","unstructured":"ISO\/IEC JTC1\/SC 42 committee. 2020. https:\/\/www.iso.org\/committee\/6794475.html. Accessed 18 Feb 2020."},{"key":"303_CR21","unstructured":"Big Data Value Association. BVA SRIA\u2014European big data value strategic research and innovation agenda. 2017. http:\/\/bdva.eu\/sites\/default\/files\/BDVA_SRIA_v4_Ed1.1.pdf. Accessed 18 Feb 2020."},{"key":"303_CR22","unstructured":"Boid D, Chang W. NIST Big Data Interoperability Framework: Volume 6, RA Version 2. NIST Big Data Program. 2018. https:\/\/bigdatawg.nist.gov\/_uploadfiles\/NIST.SP.1500-6r1.pdf. Accessed 18 Feb 2020."},{"key":"303_CR23","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1016\/j.future.2019.04.016","volume":"99","author":"I Sitt\u00f3n-Candanedo","year":"2019","unstructured":"Sitt\u00f3n-Candanedo I, Alonso RS, Corchado JM, Rodr\u00edguez-Gonz\u00e1lez S, Casado-Vara R. A review of edge computing reference architectures and a new global edge proposal. Fut Gener Comput Syst. 2019;99:278\u201394.","journal-title":"Fut Gener Comput Syst"},{"key":"303_CR24","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1145\/3331138","volume":"62","author":"A Wiedemann","year":"2019","unstructured":"Wiedemann A, Forsgren N, Wiesche M, Gewald H, Krcmar H. Research for practice: the DevOps phenomenon. Commun ACM. 2019;62:44\u20139.","journal-title":"Commun ACM"},{"key":"303_CR25","doi-asserted-by":"crossref","unstructured":"Dang Y, Lin Q, Huang P. AIOps: Real-World challenges and research innovations. In: IEEE\/ACM 41st international conference on software engineering: companion proceedings (ICSE-Companion), Montreal, Canada; 2019.","DOI":"10.1109\/ICSE-Companion.2019.00023"},{"key":"303_CR26","unstructured":"Hummer, Muthusamy V, Rausch T, Dube P, El Maghraoui K, Murthi A, Oum P. ModelOps: Cloud-based lifecycle management for reliable and trusted AI. In: IEEE international conference on cloud engineering (IC2E), Prague, Czech Republic, 24\u201327 June, 2019."},{"key":"303_CR27","doi-asserted-by":"crossref","unstructured":"Jeong H, Jeong I, Lee H, Moon S. Computation offloading for machine learning web apps in the edge server environment. In: IEEE 38th international conference on distributed computing systems, Vienna, Austria, 02\u201305 July, 2019, p. 1492\u20139.","DOI":"10.1109\/ICDCS.2018.00154"},{"key":"303_CR28","unstructured":"Jeong H, Shin CH, Lee H, Moon S. IONN: Incremental offloading of neural network computations from mobile devices to edge servers. In: ACM symposium on cloud computing, Carlsbad, CA, USA, 11\u201313 October, 2018, p. 401\u201311."},{"key":"303_CR29","doi-asserted-by":"crossref","unstructured":"Liu L, Li H, Gruteser M. Edge assisted real-time object detection for mobile augmented reality. In: The 25th annual international conference on mobile computing and networking, Los Cabos, Mexico, 21\u201325 October, 2019.","DOI":"10.1145\/3300061.3300116"},{"key":"303_CR30","doi-asserted-by":"crossref","unstructured":"Teerapittayanon S, McDanel B, Kung HT. Distributed deep neural networks over the cloud, the edge and end devices. In: IEEE 37th international conference on distributed computing systems, Atlanta, GA, USA, 05\u201308 June, 2017, p. 328\u2013339.","DOI":"10.1109\/ICDCS.2017.226"},{"key":"303_CR31","unstructured":"Zhou L, Wen H, Teodorescu R, Du DHC. Distributing deep neural networks with containerized partitions at the edge. In: 2nd usenix workshop on hot topics in edge computing, Renton, WA, USA, 09 July, 2019."},{"key":"303_CR32","doi-asserted-by":"crossref","unstructured":"Trinelli M, Gallo M, Rifai M, Pianese F. Transparent AR processing acceleration at the edge. In: 2nd international workshop on edge systems, analytics and networking, Dresden, Germany, 25 March 2019, p. 30\u201335.","DOI":"10.1145\/3301418.3313942"},{"key":"303_CR33","unstructured":"Redmon J, Farhadi A. YOLO9000: better, faster, stronger. In: The IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, 21\u201326 July, 2017, p. 7263\u201371."},{"key":"303_CR34","unstructured":"Ran X, Chen H, Chu X, Liu Z, Chen J. DeepDecision: A Mobile deep learning framework for edge video analytics. In: IEEE conference on computer communications, Honolulu, HI, USA, 16\u201319 April, 2018, p. 1421\u20139."},{"key":"303_CR35","unstructured":"Hardy C, Merrer EL, Sericola B. Distributed deep learning on edge-devices: feasibility via adaptive compression. In: IEEE 16th international symposium on network computing and applications (NCA), Cambridge, MA, USA, 30 October 30\u201301 November, 2017."},{"key":"303_CR36","unstructured":"Lim H, Andersen DG, Kaminsky M. 3LC: Lightweight and effective traffic compression for distributed machine learning. In: SysML conference, Stanford, CA, USA, 31 March\u201302 April, 2019."},{"key":"303_CR37","unstructured":"Hsieh K, Harlap A, Vijaykumar N, Konomis D, Ganger GR, Gibbons PB. Gaia: Geo-distributed machine learning approaching LAN speeds. In: The 14th USENIX symposium on networked systems design and implementation (NSDI\u201917), Boston, MA, USA, 27\u201329 March, 2017, p. 629\u201347."},{"key":"303_CR38","unstructured":"Hong R, Chandra A. DLion: Decentralized distributed deep learning in micro-clouds. In: 11th USENIX workshop on hot topics in cloud computing, Renton, WA, USA, 08 July, 2019."},{"key":"303_CR39","doi-asserted-by":"crossref","unstructured":"Li Y, Park J, Alian M, Yuan Y, Qu Z, Pan P, Wang R, Schwing AG, Esmaeilzadeh H, Kim NS. A network-centric hardware\/algorithm co-design to accelerate distributed training of deep neural networks. In: 51st annual IEEE\/ACM international symposium on microarchitecture, Fukuoka, Japan, 20\u201324 October, 2018, p. 175\u2013188.","DOI":"10.1109\/MICRO.2018.00023"},{"issue":"23","key":"303_CR40","first-page":"5324","volume":"19","author":"J Chen","year":"2019","unstructured":"Chen J, Li K, Deng Q, Li K, Yu PS. Distributed Deep Learning Model for Intelligent Video Surveillance Systems with Edge Computing. IEEE Ind Inform. 2019;19(23):5324.","journal-title":"IEEE Ind Inform."},{"key":"303_CR41","unstructured":"Li D, Salonidis D, Desai NV, Chuah MC. DeepCham: collaborative edge-mediated adaptive deep learning for mobile object recognition. In: IEEE\/ACM symposium on edge computing, Washington, DC, USA, 27\u201328 October, 2016, p. 64\u201376."},{"key":"303_CR42","unstructured":"ETSI. Multi-access edge computing (MEC). 2020. https:\/\/www.etsi.org\/technologies\/multi-access-edge-computing. Accessed 18 Feb 2020."},{"key":"303_CR43","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1080\/00031305.1993.10475983","volume":"47","author":"RM Lindsay","year":"1993","unstructured":"Lindsay RM, Ehrenberg ASC. The design of replicated studies. Am Stat. 1993;47:217\u201328.","journal-title":"Am Stat"},{"key":"303_CR44","unstructured":"Galster M, Avgeriou P. Empirically-grounded reference architectures: a proposal. In: Joint ACM SIGSOFT conference on quality of software architectures and ACM SIGSOFT conference on quality of software architectures and ACM SIGSOFT symposium on architecting critical systems, Boulder, Colorado, USA, 20\u201324 June, 2011."},{"key":"303_CR45","doi-asserted-by":"crossref","unstructured":"Kascavalci HC, G\u00f6ren S. A deep learning based distributed smart surveillance architecture using edge and CloudComputing. In: International conference on deep learning and machine learning in emerging applications (Deep-ML). Istanbul, Turkey, 26\u201328 April, 2019.","DOI":"10.1109\/Deep-ML.2019.00009"},{"key":"303_CR46","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1016\/j.future.2019.10.043","volume":"104","author":"S Tuli","year":"2020","unstructured":"Tuli S, et al. HealthFog: an ensemble deep learning based Smart Healthcare System for Automatic Diagnosis of Heart Diseases in integrated IoT and fog computing environments. Fut Gener Comput Syst. 2020;104:187\u2013200.","journal-title":"Fut Gener Comput Syst"},{"key":"303_CR47","unstructured":"The Open Group. ArchiMate 3.1 Specification. 2020. https:\/\/pubs.opengroup.org\/architecture\/archimate3-doc\/. Accessed 18 Feb 2020."},{"key":"303_CR48","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1016\/j.ins.2017.04.042","volume":"408","author":"W Wei","year":"2017","unstructured":"Wei W, Song H, Li W, Shen P, Vasilakos A. Gradient-driven parking navigation using a continuous information potential field based on wireless sensor network. Inform Sci. 2017;408:100\u201314.","journal-title":"Inform Sci"},{"key":"303_CR49","doi-asserted-by":"publisher","first-page":"2028","DOI":"10.1109\/ACCESS.2015.2490723","volume":"3","author":"A Immonen","year":"2015","unstructured":"Immonen A, P\u00e4\u00e4kk\u00f6nen P, Ovaska E. Evaluating the quality of social media data in big data architecture. IEEE Access. 2015;3:2028\u201343.","journal-title":"IEEE Access."},{"key":"303_CR50","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1186\/s40537-017-0066-7","volume":"4","author":"P P\u00e4\u00e4kk\u00f6nen","year":"2017","unstructured":"P\u00e4\u00e4kk\u00f6nen P, Jokitulppo J. Quality management architecture for social media data. J Big Data. 2017;4:6.","journal-title":"J Big Data."},{"key":"303_CR51","doi-asserted-by":"publisher","first-page":"248","DOI":"10.1016\/j.comcom.2018.07.015","volume":"129","author":"J Moysen","year":"2018","unstructured":"Moysen J, Giupponi L. From 4G to 5G: self-organized network management meets machine learning. Comput Commun. 2018;129:248\u201368.","journal-title":"Comput Commun"},{"key":"303_CR52","doi-asserted-by":"publisher","first-page":"134","DOI":"10.1109\/MWC.2019.1800498","volume":"26","author":"DM Gutierrez-Estevez","year":"2019","unstructured":"Gutierrez-Estevez DM, Gramaglia M, De Domenico A, Dandachi G, Khatibi S, Tsolkas D, Balan I, Garcia-Saavedra A, Elzur U, Wang Y. Artificial intelligence for elastic management and orchestration of 5G networks. IEEE Wirel Commun. 2019;26:134\u201341.","journal-title":"IEEE Wirel Commun"},{"key":"303_CR53","unstructured":"Google. Machine learning glossary. 2020. https:\/\/developers.google.com\/machine-learning\/glossary. Accessed 18 Feb 2020."}],"container-title":["Journal of Big Data"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-020-00303-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s40537-020-00303-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-020-00303-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,3]],"date-time":"2024-08-03T14:21:06Z","timestamp":1722694866000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-020-00303-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,6]]},"references-count":53,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["303"],"URL":"https:\/\/doi.org\/10.1186\/s40537-020-00303-y","relation":{},"ISSN":["2196-1115"],"issn-type":[{"value":"2196-1115","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,6]]},"assertion":[{"value":"27 November 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 March 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 April 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare that they have no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"25"}}