{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:13:50Z","timestamp":1760231630871,"version":"build-2065373602"},"reference-count":54,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2022,5,25]],"date-time":"2022-05-25T00:00:00Z","timestamp":1653436800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"RIE2020 Advanced Manufacturing and Engineering (AME) Programmatic Programme","award":["A19E3b0099","SRT3IS21164"],"award-info":[{"award-number":["A19E3b0099","SRT3IS21164"]}]},{"name":"SUTD Start-up","award":["A19E3b0099","SRT3IS21164"],"award-info":[{"award-number":["A19E3b0099","SRT3IS21164"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>The appealing properties of secure hardware solutions such as trusted execution environment (TEE) including low computational overhead, confidentiality guarantee, and reduced attack surface have prompted considerable interest in adopting them for secure stream processing applications. In this paper, we revisit the design of parallel stream join algorithms on multicore processors with TEEs. In particular, we conduct a series of profiling experiments to investigate the impact of alternative design choices to parallelize stream joins on TEE including: (1) execution approaches, (2) partitioning schemes, and (3) distributed scheduling strategies. From the profiling study, we observe three major high-performance impediments: (a) the computational overhead introduced with cryptographic primitives associated with page swapping operations, (b) the restrictive Enclave Page Cache (EPC) size that limits the supported amount of in-memory processing, and (c) the lack of vertical scalability to support the increasing workload often required for near real-time applications. Addressing these issues allowed us to design SecJoin, a more efficient parallel stream join algorithm that exploits modern scale-out architectures with TEEs rendering no trade-offs on security whilst optimizing performance. We present our model-driven parameterization of SecJoin and share our experimental results which have shown up to 4-folds of improvements in terms of throughput and latency.<\/jats:p>","DOI":"10.3390\/a15060183","type":"journal-article","created":{"date-parts":[[2022,5,25]],"date-time":"2022-05-25T08:41:33Z","timestamp":1653468093000},"page":"183","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Revisiting the Design of Parallel Stream Joins on Trusted Execution Environments"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7197-8028","authenticated-orcid":false,"given":"Souhail","family":"Meftah","sequence":"first","affiliation":[{"name":"College of Design and Engineering, National University of Singapore (NUS), Singapore 117575, Singapore"},{"name":"Institute of Infocomm Research, Agency for Science, Technology and Research (A*STAR), Singapore 138632, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9927-6925","authenticated-orcid":false,"given":"Shuhao","family":"Zhang","sequence":"additional","affiliation":[{"name":"Information Systems Technology and Design (ISTD), Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9000-1813","authenticated-orcid":false,"given":"Bharadwaj","family":"Veeravalli","sequence":"additional","affiliation":[{"name":"College of Design and Engineering, National University of Singapore (NUS), Singapore 117575, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5652-3455","authenticated-orcid":false,"given":"Khin Mi Mi","family":"Aung","sequence":"additional","affiliation":[{"name":"Institute of Infocomm Research, Agency for Science, Technology and Research (A*STAR), Singapore 138632, Singapore"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1809","DOI":"10.14778\/3229863.3229869","article-title":"Providing Streaming Joins as a Service at Facebook","volume":"11","author":"Lei","year":"2018","journal-title":"Proc. VLDB Endow."},{"unstructured":"Linden, T., Khandelwal, R., Harkous, H., and Fawaz, K. (2018). The Privacy Policy Landscape After the GDPR. arXiv.","key":"ref_2"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1093\/idpl\/ipx016","article-title":"Data privacy law in Singapore: The Personal Data Protection Act 2012","volume":"7","year":"2017","journal-title":"Int. Data Priv. Law"},{"unstructured":"Najafi, M., Sadoghi, M., and Jacobsen, H.A. (2016, January 22\u201324). SplitJoin: A Scalable, Low-latency Stream Join Architecture with Adjustable Ordering Precision. Proceedings of the 2016 USENIX Annual Technical Conference (USENIX ATC 16), Denver, CO, USA.","key":"ref_4"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"709","DOI":"10.14778\/2732939.2732944","article-title":"Low-Latency Handshake Join","volume":"7","author":"Roy","year":"2014","journal-title":"Proc. VLDB Endow."},{"doi-asserted-by":"crossref","unstructured":"Teubner, J., and Mueller, R. (2011). How Soccer Players Would Do Stream Joins, Association for Computing Machinery.","key":"ref_6","DOI":"10.1145\/1989323.1989389"},{"doi-asserted-by":"crossref","unstructured":"Shahvarani, A., and Jacobsen, H.A. (2020). Parallel Index-Based Stream Join on a Multicore CPU, Association for Computing Machinery.","key":"ref_7","DOI":"10.1145\/3318464.3380576"},{"doi-asserted-by":"crossref","unstructured":"Behzadnia, P. (2021, January 14\u201317). Shared-Memory Parallel Hash-Based Stream Join in Continuous Data Streams. Proceedings of the DEXA, Bratislava, Slovakia.","key":"ref_8","DOI":"10.1007\/978-3-030-86475-0_30"},{"doi-asserted-by":"crossref","unstructured":"Lin, Q., Ooi, B.C., Wang, Z., and Yu, C. (2015). Scalable Distributed Stream Join Processing, Association for Computing Machinery.","key":"ref_9","DOI":"10.1145\/2723372.2746485"},{"unstructured":"Herschel, M., Galhardas, H., Reinwald, B., Fundulaki, I., Binnig, C., and Kaoudi, Z. (2019). Streaming HyperCube: A Massively Parallel Stream Join Algorithm, EDBT.","key":"ref_10"},{"unstructured":"Mast, K., Chen, L., and Sirer, E. (2018). Enabling Strong Database Integrity using Trusted Execution Environments. arXiv.","key":"ref_11"},{"unstructured":"(2022, May 23). EdgelessDB\u2014The SQL Database Tailor-Made for Confidential Computing. Available online: https:\/\/www.edgeless.systems\/products\/edgelessdb\/.","key":"ref_12"},{"unstructured":"Han, Z., and Hu, H. (2022, May 23). ProDB: A Memory-Secure Database Using Hardware Enclave and Practical Oblivious RAM. Available online: https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0306437920301332?via%3Dihub.","key":"ref_13"},{"unstructured":"Ribeiro, P.S., Santos, N., and Duarte, N.O. (2018). DBStore: A TrustZone-Backed Database Management System for Mobile Applications, ICETE.","key":"ref_14"},{"unstructured":"Zheng, W., Dave, A., Beekman, J.G., Popa, R.A., Gonzalez, J.E., and Stoica, I. (2017, January 27\u201329). Opaque: An Oblivious and Encrypted Distributed Analytics Platform. Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation, Boston, MA, USA.","key":"ref_15"},{"doi-asserted-by":"crossref","unstructured":"Eskandarian, S., and Zaharia, M. (2019). ObliDB: Oblivious Query Processing for Secure Databases. arXiv.","key":"ref_16","DOI":"10.14778\/3364324.3364331"},{"unstructured":"Nilsson, A., Bideh, P.N., and Brorsson, J. (2020). A Survey of Published Attacks on Intel SGX. arXiv.","key":"ref_17"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1109\/MSEC.2019.2947124","article-title":"Trusted Execution Environments: Properties, Applications, and Challenges","volume":"18","author":"Jauernig","year":"2020","journal-title":"IEEE Secur. Priv."},{"unstructured":"Brasser, F., M\u00fcller, U., Dmitrienko, A., Kostiainen, K., Capkun, S., and Sadeghi, A.R. (2017, January 14\u201315). Software Grand Exposure: SGX Cache Attacks Are Practical. Proceedings of the 11th USENIX Workshop on Offensive Technologies (WOOT 17), Vancouver, BC, Canada.","key":"ref_19"},{"unstructured":"Canella, C., Bulck, J.V., Schwarz, M., Lipp, M., von Berg, B., Ortner, P., Piessens, F., Evtyushkin, D., and Gruss, D. (2019, January 14\u201316). A Systematic Evaluation of Transient Execution Attacks and Defenses. Proceedings of the 28th USENIX Security Symposium (USENIX Security 19), Santa Clara, CA, USA.","key":"ref_20"},{"unstructured":"Costan, V., and Devadas, S. (2021, October 03). Intel SGX Explained. Available online: https:\/\/eprint.iacr.org\/2016\/086.","key":"ref_21"},{"doi-asserted-by":"crossref","unstructured":"Garay, J.A., and Gennaro, R. (2014). Algorithms in HElib. Advances in Cryptology\u2014CRYPTO 2014, Springer.","key":"ref_22","DOI":"10.1007\/978-3-662-44381-1"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"3740","DOI":"10.1109\/TIFS.2021.3090959","article-title":"DOReN: Toward Efficient Deep Convolutional Neural Networks with Fully Homomorphic Encryption","volume":"16","author":"Meftah","year":"2021","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/j.future.2022.03.033","article-title":"Towards high performance homomorphic encryption for inference tasks on CPU: An MPI approach","volume":"134","author":"Meftah","year":"2022","journal-title":"Future Gener. Comput. Syst."},{"doi-asserted-by":"crossref","unstructured":"Castro Fernandez, R., Migliavacca, M., Kalyvianaki, E., and Pietzuch, P. (2013, January 22\u201327). Integrating Scale out and Fault Tolerance in Stream Processing Using Operator State Management. Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, New York, NY, USA.","key":"ref_25","DOI":"10.1145\/2463676.2465282"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"441","DOI":"10.14778\/2732279.2732281","article-title":"Scalable and Adaptive Online Joins","volume":"7","author":"Elseidy","year":"2014","journal-title":"Proc. VLDB Endow."},{"unstructured":"Wilschut, A.N., and Apers, P.M.G. (1991, January 4\u20136). Dataflow query execution in a parallel main-memory environment. Proceedings of the First International Conference on Parallel and Distributed Information Systems, Miami Beach, FL, USA.","key":"ref_27"},{"doi-asserted-by":"crossref","unstructured":"Dittrich, J.P., Seeger, B., Taylor, D.S., and Widmayer, P. (2002, January 20\u201323). Progressive Merge Join: A Generic and Non-blocking Sort-based Join Algorithm. Proceedings of the 28th International Conference on Very Large Data Bases, VLDB Endowment, Hong Kong, China.","key":"ref_28","DOI":"10.1016\/B978-155860869-6\/50034-2"},{"unstructured":"Urhan, T., and Franklin, M.J. (2000). Xjoin: A Reactively-Scheduled Pipelined Join Operator \u00eb, IEEE.","key":"ref_29"},{"unstructured":"Mokbel, M.F., Lu, M., and Aref, W.G. (2004, January 2). Hash-Merge Join: A Non-blocking Join Algorithm for Producing Fast and Early Join Results. Proceedings of the 20th International Conference on Data Engineering, Boston, MA, USA.","key":"ref_30"},{"unstructured":"Lawrence, R. (September, January 30). Early Hash Join: A Configurable Algorithm for the Efficient and Early Production of Join Results. Proceedings of the 31st International Conference on Very Large Data Bases, Trondheim, Norway.","key":"ref_31"},{"doi-asserted-by":"crossref","unstructured":"Tao, Y., Yiu, M.L., Papadias, D., Hadjieleftheriou, M., and Mamoulis, N. (2005, January 14\u201316). RPJ: Producing Fast Join Results on Streams Through Rate-based Optimization. Proceedings of the 2005 ACM SIGMOD International Conference on Management of Data, Baltimore, MD, USA.","key":"ref_32","DOI":"10.1145\/1066157.1066200"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1145\/304181.304208","article-title":"Ripple Joins for Online Aggregation","volume":"28","author":"Haas","year":"1999","journal-title":"SIGMOD Rec."},{"doi-asserted-by":"crossref","unstructured":"Li, F., Wu, B., Yi, K., and Zhao, Z. (July, January 26). Wander Join: Online Aggregation via Random Walks. Proceedings of the 2016 International Conference on Management of Data, San Francisco, CA, USA.","key":"ref_34","DOI":"10.1145\/2882903.2915235"},{"unstructured":"Kaplan, D. (2016). AMD x86 Memory Encryption Technologies, USENIX Association.","key":"ref_35"},{"unstructured":"Alves, T., and Felton, D. (2021, October 03). Trustzone: Integrated Hardware and Software Security. Available online: https:\/\/www.techonline.com\/tech-papers\/trustzone-integrated-hardware-and-software-security\/.","key":"ref_36"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1109\/MM.2019.2897677","article-title":"On the Spectre and Meltdown Processor Security Vulnerabilities","volume":"39","author":"Hill","year":"2019","journal-title":"IEEE Micro"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"353","DOI":"10.14778\/2735496.2735499","article-title":"Memory-Efficient Hash Joins","volume":"8","author":"Barber","year":"2014","journal-title":"Proc. VLDB Endow."},{"doi-asserted-by":"crossref","unstructured":"Blanas, S., Li, Y., and Patel, J.M. (2011, January 12\u201316). Design and Evaluation of Main Memory Hash Join Algorithms for Multi-Core CPUs. Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data, Athens, Greece.","key":"ref_39","DOI":"10.1145\/1989323.1989328"},{"doi-asserted-by":"crossref","unstructured":"Taassori, M., Shafiee, A., and Balasubramonian, R. (2018, January 24\u201328). VAULT: Reducing Paging Overheads in SGX with Efficient Integrity Verification Structures. Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems, Williamsburg, VA, USA.","key":"ref_40","DOI":"10.1145\/3173162.3177155"},{"unstructured":"Graefe, G. (1994, January 14\u201318). Sort-merge-join: An idea whose time has(h) passed?. Proceedings of the 1994 IEEE 10th International Conference on Data Engineering, Washington, DC, USA.","key":"ref_41"},{"doi-asserted-by":"crossref","unstructured":"Brenner, S., Wulf, C., Goltzsche, D., Weichbrodt, N., Lorenz, M., Fetzer, C., Pietzuch, P., and Kapitza, R. (2016, January 12\u201316). SecureKeeper: Confidential ZooKeeper Using Intel SGX. Proceedings of the 17th International Middleware Conference, Trento, Italy.","key":"ref_42","DOI":"10.1145\/2988336.2988350"},{"doi-asserted-by":"crossref","unstructured":"Zhao, C., Saifuding, D., Tian, H., Zhang, Y., and Xing, C. (2016, January 23\u201325). On the Performance of Intel SGX. Proceedings of the 13th Web Information Systems and Applications Conference (WISA), Wuhan, China.","key":"ref_43","DOI":"10.1109\/WISA.2016.45"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3322205.3311076","article-title":"Everything You Should Know About Intel SGX Performance on Virtualized Systems","volume":"3","author":"Bui","year":"2019","journal-title":"Proc. ACM Meas. Anal. Comput. Syst."},{"unstructured":"(2021, October 03). pmbw\u2014Parallel Memory Bandwidth Benchmark\/Measure. Available online: https:\/\/github.com\/bingmann\/pmbw.","key":"ref_45"},{"unstructured":"Che Tsai, C., Porter, D.E., and Vij, M. (2017, January 12\u201314). Graphene-SGX: A Practical Library OS for Unmodified Applications on SGX. Proceedings of the 2017 USENIX Annual Technical Conference (USENIX ATC 17), Santa Clara, CA, USA.","key":"ref_46"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1378","DOI":"10.14778\/1687553.1687564","article-title":"Sort vs. Hash Revisited: Fast Join Implementation on Modern Multi-Core CPUs","volume":"2","author":"Kim","year":"2009","journal-title":"Proc. VLDB Endow."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"789","DOI":"10.1016\/0167-8191(96)00024-5","article-title":"A high-performance, portable implementation of the MPI message passing interface standard","volume":"22","author":"Gropp","year":"1996","journal-title":"Parallel Comput."},{"unstructured":"Peng, I.B., Markidis, S., Gioiosa, R., Kestor, G., and Laure, E. (2017). MPI Streams for HPC Applications. arXiv.","key":"ref_49"},{"doi-asserted-by":"crossref","unstructured":"Karimov, J., Rabl, T., Katsifodimos, A., Samarev, R., Heiskanen, H., and Markl, V. (201, January 16\u201319). Benchmarking distributed stream data processing systems. Proceedings of the 2018 IEEE 34th International Conference on Data Engineering, Paris, France.","key":"ref_50","DOI":"10.1109\/ICDE.2018.00169"},{"doi-asserted-by":"crossref","unstructured":"Chintapalli, S., Dagit, D., Evans, B., Farivar, R., Graves, T., Holderbaugh, M., Liu, Z., Nusbaum, K., Patil, K., and Peng, B.J. (2016, January 23\u201327). Benchmarking streaming computation engines: Storm, flink and spark streaming. Proceedings of the 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Chicago, IL, USA.","key":"ref_51","DOI":"10.1109\/IPDPSW.2016.138"},{"doi-asserted-by":"crossref","unstructured":"Gulisano, V., Jerzak, Z., Voulgaris, S., and Ziekow, H. (2016, January 20\u201324). The DEBS 2016 grand challenge. Proceedings of the 10th ACM International Conference on Distributed and Event-Based Systems, Irvine, CA, USA.","key":"ref_52","DOI":"10.1145\/2933267.2933519"},{"doi-asserted-by":"crossref","unstructured":"McKeen, F., Alexandrovich, I., Anati, I., Caspi, D., Johnson, S., Leslie-Hurd, R., and Rozas, C. (2016, January 18). Intel\u00ae Software Guard Extensions (Intel\u00ae SGX) Support for Dynamic Memory Management Inside an Enclave. Proceedings of the Hardware and Architectural Support for Security and Privacy 2016, Seoul, Korea.","key":"ref_53","DOI":"10.1145\/2948618.2954331"},{"doi-asserted-by":"crossref","unstructured":"Peng, I.B., Markidis, S., Laure, E., Holmes, D., and Bull, M. (2015, January 15). A Data Streaming Model in MPI. Proceedings of the 3rd Workshop on Exascale MPI, Texas, AX, USA.","key":"ref_54","DOI":"10.1145\/2831129.2831131"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/15\/6\/183\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:18:23Z","timestamp":1760138303000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/15\/6\/183"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,25]]},"references-count":54,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["a15060183"],"URL":"https:\/\/doi.org\/10.3390\/a15060183","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2022,5,25]]}}}