{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T06:51:59Z","timestamp":1782370319516,"version":"3.54.5"},"reference-count":44,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T00:00:00Z","timestamp":1782259200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Limited by the computational performance limits of the CPU(Central Processing Unit), the traditional Spark architecture struggles to achieve high throughput and low latency under the dual pressure of a large data scale and real-time requirements in centralized control systems. This work uses a publicly available CNC(Computer Numerical Control) milling dataset as a functional validation proxy for time-series data processing, then extends validation to a large-scale synthetic power transmission grid dataset. Furthermore, Spark-GPU(Graphics Processing Unit) collaboration suffers from load balancing failure due to heterogeneous resource scheduling and communication overhead, thus failing to unleash its performance potential. This paper proposes a Spark-GPU fusion acceleration technology path. The path consists of three key components: first, it integrates the RAPIDS accelerator; second, it designs a GPU-aware partitioning and task co-scheduling strategy; and third, it optimizes the zero-copy data path. Together, these components realize an integrated collaboration of heterogeneous resources. Validation on real-world datasets yields the following results. In real-time aggregation scenarios, the proposed solution improves throughput by a factor of 3.7 over the pure CPU baseline and reduces end-to-end latency by 62%. Compared with the basic GPU solution, GPU utilization rises from 51.7% to 72.3%, representing a relative improvement of 39.8%. Furthermore, the solution meets industrial-grade high availability requirements. This research significantly improves the processing throughput and reduces end-to-end latency in typical centralized control scenarios, thus providing a feasible technical route for demanding concurrent centralized control scenarios such as electric power industry manufacturing with high real-time demands.<\/jats:p>","DOI":"10.3390\/info17070625","type":"journal-article","created":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T05:48:16Z","timestamp":1782366496000},"page":"625","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Performance Optimization of Distributed Data Processing in Centralized Control System Based on Spark and GPU Collaboration"],"prefix":"10.3390","volume":"17","author":[{"given":"Xunting","family":"Wang","sequence":"first","affiliation":[{"name":"State Grid Anhui Electric Power Research Institute, Hefei 230601, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cheng","family":"Xie","sequence":"additional","affiliation":[{"name":"Anhui Electric Power Company Limited, Hefei 230022, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinjin","family":"Ding","sequence":"additional","affiliation":[{"name":"State Grid Anhui Electric Power Research Institute, Hefei 230601, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bin","family":"Xu","sequence":"additional","affiliation":[{"name":"State Grid Anhui Electric Power Research Institute, Hefei 230601, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianlin","family":"Li","sequence":"additional","affiliation":[{"name":"Anhui Electric Power Company Limited, Hefei 230022, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weimin","family":"Huang","sequence":"additional","affiliation":[{"name":"Anhui Electric Power Company Limited, Hefei 230022, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,6,24]]},"reference":[{"key":"ref_1","first-page":"16","article-title":"The fourth industrial revolution: A primer on industry 4.0 and its transformative impact","volume":"2","author":"George","year":"2024","journal-title":"Partn. Univers. Innov. Res. Publ."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Akanbi, A., and Masinde, M. (2020). A distributed stream processing middleware framework for real-time analysis of heterogeneous data on big data platform: Case of environmental monitoring. Sensors, 20.","DOI":"10.3390\/s20113166"},{"key":"ref_3","first-page":"377","article-title":"Performance evaluation of GPU-and cluster-computing for parallelization of compute-intensive tasks","volume":"17","author":"Keller","year":"2021","journal-title":"Int. J. Web Inf. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Pacella, M., Papa, A., Papadia, G., and Fedeli, E. (2025). A scalable framework for sensor data ingestion and real-time processing in cloud manufacturing. Algorithms, 18.","DOI":"10.3390\/a18010022"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Al Jawarneh, I.M., Rosa, L., Venanzi, R., Foschini, L., and Bellavista, P. (2025). Efficient Parallel Processing of Big Data on Supercomputers for Industrial IoT Environments. Electronics, 14.","DOI":"10.3390\/electronics14132626"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3390\/bdcc8010001","article-title":"Distributed Bayesian inference for large-scale IoT systems","volume":"8","author":"Vlachou","year":"2023","journal-title":"Big Data Cogn. Comput."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"23022","DOI":"10.1109\/ACCESS.2020.2970118","article-title":"Internet of things (IoT) for next-generation smart systems: A review of current challenges, future trends and prospects for emerging 5G-IoT scenarios","volume":"8","author":"Shafique","year":"2020","journal-title":"IEEE Access"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1282","DOI":"10.1109\/JPROC.2021.3087029","article-title":"Recent developments in parallel and distributed computing for remotely sensed big data processing","volume":"109","author":"Wu","year":"2021","journal-title":"Proc. IEEE"},{"key":"ref_9","first-page":"71","article-title":"A survey on spark ecosystem: Big data processing infrastructure, machine learning, and applications","volume":"34","author":"Tang","year":"2020","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"441","DOI":"10.14778\/3632093.3632107","article-title":"Gpu database systems characterization and optimization","volume":"17","author":"Cao","year":"2023","journal-title":"Proc. VLDB Endow."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Dai, F., Hossain, M.A., and Wang, Y. (2025). State of the art in parallel and distributed systems: Emerging trends and challenges. Electronics, 14.","DOI":"10.3390\/electronics14040677"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"5660","DOI":"10.1109\/TMC.2023.3312051","article-title":"Bottleneck identification in cloudified mobile networks based on distributed telemetry","volume":"23","author":"Fida","year":"2023","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"S700","DOI":"10.1137\/15M1026407","article-title":"Vispark: GPU-accelerated distributed visual computing using spark","volume":"38","author":"Choi","year":"2016","journal-title":"SIAM J. Sci. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/MCAS.2021.3071608","article-title":"Fpga acceleration for big data analytics: Challenges and opportunities","volume":"21","author":"Hoozemans","year":"2021","journal-title":"IEEE Circuits Syst. Mag."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3485126","article-title":"Query processing on heterogeneous CPU\/GPU systems","volume":"55","author":"Rosenfeld","year":"2022","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"4709","DOI":"10.14778\/3704965.3704977","article-title":"Scaling your hybrid CPU-GPU DBMS to multiple gpus","volume":"17","author":"Yogatama","year":"2024","journal-title":"Proc. VLDB Endow."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"9943289","DOI":"10.1155\/2021\/9943289","article-title":"Accelerating Spark-Based Applications with MPI and OpenACC","volume":"2021","author":"Alshahrani","year":"2021","journal-title":"Complexity"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3165","DOI":"10.1007\/s11227-020-03390-z","article-title":"Ignite-GPU: A GPU-enabled in-memory computing architecture on clusters","volume":"77","author":"Sojoodi","year":"2021","journal-title":"J. Supercomput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3589980","article-title":"Towards Accelerating Data Intensive Application\u2019s Shuffle Process Using SmartNICs","volume":"7","author":"Lin","year":"2023","journal-title":"Proc. ACM Meas. Anal. Comput. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"6119","DOI":"10.1007\/s10586-024-04267-z","article-title":"Design and realization of hybrid resource management system for heterogeneous cluster","volume":"27","author":"He","year":"2024","journal-title":"Clust. Comput."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Capel, M.I., Salguero-Hidalgo, A., and Holgado-Terriza, J.A. (2024). Parallel PSO for Efficient Neural Network Training Using GPGPU and Apache Spark in Edge Computing Sets. Algorithms, 17.","DOI":"10.20944\/preprints202407.1300.v1"},{"key":"ref_22","unstructured":"Li, P., Luo, Y., Zhang, N., and Cao, Y. (2015). Heterospark: A heterogeneous cpu\/gpu spark platform for machine learning algorithms. 2015 IEEE International Conference on Networking, Architecture and Storage (NAS), IEEE."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Chiriac, B.N., Anton, F.D., Ioni\u021b\u0103, A.D., and Vasilic\u0103, B.V. (2024). A modular AI-driven intrusion detection system for network traffic monitoring in industry 4.0, using nvidia morpheus and generative adversarial networks. Sensors, 25.","DOI":"10.3390\/s25010130"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1663","DOI":"10.1109\/TPDS.2016.2627558","article-title":"Leveraging adaptive I\/O to optimize collective data shuffling patterns for big data analytics","volume":"28","author":"Nicolae","year":"2016","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"4935","DOI":"10.1109\/TPDS.2022.3208082","article-title":"Co-concurrency mechanism for multi-GPUs in distributed heterogeneous environments","volume":"33","author":"Zhang","year":"2022","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.future.2023.10.022","article-title":"GPARS: Graph predictive algorithm for efficient resource scheduling in heterogeneous GPU clusters","volume":"152","author":"Wang","year":"2024","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"21792","DOI":"10.1109\/JIOT.2022.3181997","article-title":"Task scheduling for spark applications with data affinity on heterogeneous clusters","volume":"9","author":"Zhang","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1657","DOI":"10.1109\/TPDS.2024.3431611","article-title":"Springald: GPU-Accelerated Window-Based Aggregates Over Out-of-Order Data Streams","volume":"35","author":"Mencagli","year":"2024","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_29","first-page":"1344","article-title":"AEML: An acceleration engine for multi-GPU load-balancing in distributed heterogeneous environment","volume":"71","author":"Tang","year":"2021","journal-title":"IEEE Trans. Comput."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1186\/s40537-019-0273-5","article-title":"Graphical flow-based spark programming","volume":"7","author":"Mahapatra","year":"2020","journal-title":"J. Big Data"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1450","DOI":"10.1109\/TPDS.2023.3235439","article-title":"RTGPU: Real-time GPU scheduling of hard deadline parallel tasks with fine-grain utilization","volume":"34","author":"Zou","year":"2023","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"134110","DOI":"10.1063\/5.0018516","article-title":"Heterogeneous parallelization and acceleration of molecular dynamics simulations in GROMACS","volume":"153","author":"Pall","year":"2020","journal-title":"J. Chem. Phys."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1109\/TPDS.2021.3079202","article-title":"Horus: Interference-aware and prediction-based scheduling in deep learning systems","volume":"33","author":"Yeung","year":"2021","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3511211","article-title":"Unified holistic memory management supporting multiple big data processing frameworks over hybrid memories","volume":"39","author":"Chen","year":"2022","journal-title":"ACM Trans. Comput. Syst. (TOCS)"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"3401","DOI":"10.1364\/BOE.459198","article-title":"PCIe-based FPGA-GPU heterogeneous computation for real-time multi-emitter fitting in super-resolution localization microscopy","volume":"13","author":"Gui","year":"2022","journal-title":"Biomed. Opt. Express"},{"key":"ref_36","first-page":"3844","article-title":"A Survey of Spark Scheduling Strategy Optimization Techniques and Development Trends","volume":"83","author":"Li","year":"2025","journal-title":"Comput. Mater. Contin."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1007\/s42514-025-00225-2","article-title":"Optimization of fault tolerance for iterative graph algorithm in spark GraphX based on high performance computing cluster","volume":"7","author":"He","year":"2025","journal-title":"CCF Trans. High Perform. Comput."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"866","DOI":"10.1109\/TC.2021.3064352","article-title":"Toward qos-awareness and improved utilization of spatial multitasking gpus","volume":"71","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Comput."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2823","DOI":"10.1109\/TPDS.2021.3078254","article-title":"An incremental iterative acceleration architecture in distributed heterogeneous environments with GPUs for deep learning","volume":"32","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"e13368","DOI":"10.1016\/j.heliyon.2023.e13368","article-title":"Framing Apache Spark in life sciences","volume":"9","author":"Manconi","year":"2023","journal-title":"Heliyon"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1007\/s13222-021-00387-7","article-title":"Collecting and visualizing data lineage of Spark jobs: Digesting Spark execution plans to surface lineage graphs via a full-stack application","volume":"21","author":"Schoenenwald","year":"2021","journal-title":"Datenbank-Spektrum"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1007\/s10619-019-07272-z","article-title":"Tpstream: Low-latency and high-throughput temporal pattern matching on event streams","volume":"39","author":"Korber","year":"2021","journal-title":"Distrib. Parallel Databases"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3747841","article-title":"Cppless: Single-Source and High-Performance Serverless Programming in C++","volume":"22","author":"Copik","year":"2025","journal-title":"ACM Trans. Archit. Code Optim."},{"key":"ref_44","first-page":"20","article-title":"A review on shared resource contention in multicores and its mitigating techniques","volume":"9","author":"Jain","year":"2020","journal-title":"Int. J. High Perform. Syst. Archit."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/17\/7\/625\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T06:45:07Z","timestamp":1782369907000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/17\/7\/625"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,24]]},"references-count":44,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2026,7]]}},"alternative-id":["info17070625"],"URL":"https:\/\/doi.org\/10.3390\/info17070625","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,24]]}}}