{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T15:03:54Z","timestamp":1779203034075,"version":"3.51.4"},"reference-count":59,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2024,8]]},"abstract":"<jats:p>Data Governance is an increasingly critical feature of modern cloud database systems, enabling administrators to set granular access policies on their data. AWS customers want to define row or column filtering on their blob storage data and access it using popular tools such as Apache Spark. AWS EMR provides a managed and serverless solution that lets users run Spark jobs in the AWS cloud with imperative and declarative programming against their data, while securely enforcing the fine-grained access controls defined on those datasets. Spark runs its compiler and scheduler alongside the user application and embeds user-defined functions in query plans, giving a threat actor direct access to its memory space. This introduces attack vectors such as information disclosure or privilege escalation during policy enforcement, in addition to well-researched threats such as SQL side channel attacks. In this paper, we present Membrane: a novel approach to secure query plans with declarative and imperative code. The innovation comes from splitting the Spark driver in two in order to rewrite query plans with security boundaries while avoiding traditional tradeoffs when using container isolation techniques. The approach described herein enables applying fine grained data access controls to both SQL and map-reduce Spark jobs, with negligible performance and cost differences.<\/jats:p>","DOI":"10.14778\/3685800.3685808","type":"journal-article","created":{"date-parts":[[2024,11,8]],"date-time":"2024-11-08T17:25:21Z","timestamp":1731086721000},"page":"3813-3826","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Membrane - Safe and Performant Data Access Controls in Apache Spark in the Presence of Imperative Code"],"prefix":"10.14778","volume":"17","author":[{"given":"Andrei","family":"Paduroiu","sequence":"first","affiliation":[{"name":"Amazon Web Services"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sungheun","family":"Wi","sequence":"additional","affiliation":[{"name":"Amazon Web Services"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yan","family":"Yan","sequence":"additional","affiliation":[{"name":"Amazon Web Services"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Roni","family":"Burd","sequence":"additional","affiliation":[{"name":"Amazon Web Services"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ruhollah","family":"Farchtchi","sequence":"additional","affiliation":[{"name":"Amazon Web Services"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Giovanni Matteo","family":"Fumarola","sequence":"additional","affiliation":[{"name":"Amazon Web Services"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,11,8]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2005.64"},{"key":"e_1_2_1_2_1","volume-title":"Accessed","year":"2024","unstructured":"Amazon. 2024. Amazon Elastic Kubernetes Service. https:\/\/aws.amazon.com\/eks. Accessed: January 22, 2024."},{"key":"e_1_2_1_3_1","volume-title":"Accessed","year":"2024","unstructured":"Amazon. 2024. AWS Fargate. https:\/\/aws.amazon.com\/fargate. Accessed: January 22, 2024."},{"key":"e_1_2_1_4_1","volume-title":"Big Data Platform - Amazon EMR - AWS. https:\/\/aws.amazon.com\/emr. Accessed","year":"2024","unstructured":"Amazon. 2024. Big Data Platform - Amazon EMR - AWS. https:\/\/aws.amazon.com\/emr. Accessed: February 5, 2024."},{"key":"e_1_2_1_5_1","volume-title":"Cloud Compute Capacity - Amazon EC2 - AWS. https:\/\/aws.amazon.com\/ec2. Accessed","year":"2024","unstructured":"Amazon. 2024. Cloud Compute Capacity - Amazon EC2 - AWS. https:\/\/aws.amazon.com\/ec2. Accessed: February 5, 2024."},{"key":"e_1_2_1_6_1","volume-title":"Cloud Object Storage - Amazon S3 - AWS. https:\/\/aws.amazon.com\/s3. Accessed","year":"2024","unstructured":"Amazon. 2024. Cloud Object Storage - Amazon S3 - AWS. https:\/\/aws.amazon.com\/s3. Accessed: February 5, 2024."},{"key":"e_1_2_1_7_1","volume-title":"Accessed","year":"2024","unstructured":"Amazon. 2024. Data Catalog and crawlers in AWS Glue. https:\/\/docs.aws.amazon.com\/glue\/latest\/dg\/catalog-and-crawler.html. Accessed: February 27, 2024."},{"key":"e_1_2_1_8_1","volume-title":"Open-Source Big Data Analytics | Amazon EMR Serverless | Amazon Web Services. https:\/\/aws.amazon.com\/emr\/serverless. Accessed","year":"2024","unstructured":"Amazon. 2024. Open-Source Big Data Analytics | Amazon EMR Serverless | Amazon Web Services. https:\/\/aws.amazon.com\/emr\/serverless. Accessed: February 5, 2024."},{"key":"e_1_2_1_9_1","volume-title":"Secure Data Lake - AWS Lake Formation - AWS. https:\/\/aws.amazon.com\/lake-formation. Accessed","year":"2024","unstructured":"Amazon. 2024. Secure Data Lake - AWS Lake Formation - AWS. https:\/\/aws.amazon.com\/lake-formation. Accessed: February 5, 2024."},{"key":"e_1_2_1_10_1","volume-title":"Accessed","year":"2024","unstructured":"Amazon. 2024. Serverless and Containers - Logical Separation on AWS. https:\/\/docs.aws.amazon.com\/whitepapers\/latest\/logical-separation\/serverless-and-containers.html. Accessed: January 29, 2024."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2588555.2595640"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2742797"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3460120.3484751"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/275487.275492"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588943"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1327452.1327492"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.5220\/0005858401960206"},{"key":"e_1_2_1_18_1","volume-title":"Accessed","author":"Foundation Apache Software","year":"2023","unstructured":"Apache Software Foundation. 2023. Spark Configuration - Stage Level Scheduling Overview. https:\/\/spark.apache.org\/docs\/3.5.0\/configuration.html#stage-level-scheduling-overview. Accessed: January 29, 2024."},{"key":"e_1_2_1_19_1","volume-title":"Accessed","author":"Foundation Apache Software","year":"2023","unstructured":"Apache Software Foundation. 2023. Spark Connect. https:\/\/spark.apache.org\/docs\/3.5.0\/spark-connect-overview.html. Accessed: February 14, 2024."},{"key":"e_1_2_1_20_1","volume-title":"Accessed","author":"Foundation Apache Software","year":"2024","unstructured":"Apache Software Foundation. 2024. Apache Hadoop YARN. https:\/\/hadoop.apache.org\/docs\/stable\/hadoop-yarn\/hadoop-yarn-site\/YARN.html. Accessed: January 22, 2024."},{"key":"e_1_2_1_21_1","volume-title":"https:\/\/hive.apache.org. Accessed","author":"Foundation Apache Software","year":"2024","unstructured":"Apache Software Foundation. 2024. Apache Hive. https:\/\/hive.apache.org. Accessed: February 5, 2024."},{"key":"e_1_2_1_22_1","volume-title":"Accessed","author":"Foundation Apache Software","year":"2024","unstructured":"Apache Software Foundation. 2024. Apache Iceberg. https:\/\/iceberg.apache.org. Accessed: February 27, 2024."},{"key":"e_1_2_1_23_1","volume-title":"Accessed","author":"Foundation Apache Software","year":"2024","unstructured":"Apache Software Foundation. 2024. Apache Mesos. https:\/\/mesos.apache.org. Accessed: January 26, 2024."},{"key":"e_1_2_1_24_1","volume-title":"Accessed","author":"Foundation Apache Software","year":"2024","unstructured":"Apache Software Foundation. 2024. Apache Ranger. https:\/\/ranger.apache.org. Accessed: February 16, 2024."},{"key":"e_1_2_1_25_1","volume-title":"Accessed","author":"Foundation Apache Software","year":"2024","unstructured":"Apache Software Foundation. 2024. Apache Sentry. https:\/\/sentry.apache.org. Accessed: February 16, 2024."},{"key":"e_1_2_1_26_1","volume-title":"https:\/\/spark.apache.org. Accessed","author":"Foundation Apache Software","year":"2024","unstructured":"Apache Software Foundation. 2024. Apache Spark. https:\/\/spark.apache.org. Accessed: February 5, 2024."},{"key":"e_1_2_1_27_1","volume-title":"Accessed","author":"Foundation Apache Software","year":"2024","unstructured":"Apache Software Foundation. 2024. Scalar User Defined Aggregate Functions (UDAFs). https:\/\/spark.apache.org\/docs\/latest\/sql-ref-functions-udf-aggregate.html. Accessed: January 29, 2024."},{"key":"e_1_2_1_28_1","volume-title":"Accessed","author":"Foundation Apache Software","year":"2024","unstructured":"Apache Software Foundation. 2024. Scalar User Defined Functions (UDFs). https:\/\/spark.apache.org\/docs\/latest\/sql-ref-functions-udf-scalar.html. Accessed: January 29, 2024."},{"key":"e_1_2_1_29_1","volume-title":"Accessed","author":"Foundation Apache Software","year":"2024","unstructured":"Apache Software Foundation. 2024. Spark basicLogicalOperators.scala. https:\/\/github.com\/apache\/spark\/blob\/branch-3.5\/sql\/catalyst\/src\/main\/scala\/org\/apache\/spark\/sql\/catalyst\/plans\/logical\/basicLogicalOperators.scala#L316. Accessed: January 29, 2024."},{"key":"e_1_2_1_30_1","volume-title":"Accessed","author":"Foundation Apache Software","year":"2024","unstructured":"Apache Software Foundation. 2024. Spark Cluster Mode Overview. https:\/\/spark.apache.org\/docs\/latest\/cluster-overview.html. Accessed: January 29, 2024."},{"key":"e_1_2_1_31_1","volume-title":"Accessed","author":"Foundation Apache Software","year":"2024","unstructured":"Apache Software Foundation. 2024. Spark QueryExecution.scala. https:\/\/github.com\/apache\/spark\/blob\/branch-3.5\/sql\/core\/src\/main\/scala\/org\/apache\/spark\/sql\/execution\/QueryExecution.scala. Accessed: January 29, 2024."},{"key":"e_1_2_1_32_1","volume-title":"Accessed","author":"Foundation Apache Software","year":"2024","unstructured":"Apache Software Foundation. 2024. Spark UserDefinedType. https:\/\/spark.apache.org\/docs\/latest\/api\/java\/org\/apache\/spark\/sql\/types\/UserDefinedType.html. Accessed: January 22, 2024."},{"key":"e_1_2_1_33_1","volume-title":"https:\/\/trino.io. Accessed","author":"Foundation Trino Software","year":"2024","unstructured":"Trino Software Foundation. 2024. Trino | Distributed SQL query engine for big data. https:\/\/trino.io. Accessed: February 5, 2024."},{"key":"e_1_2_1_34_1","volume-title":"8th USENIX Symposium on Networked Systems Design and Implementation (NSDI 11)","author":"Hindman Benjamin","year":"2011","unstructured":"Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony D. Joseph, Randy Katz, Scott Shenker, and Ion Stoica. 2011. Mesos: A Platform for {Fine-Grained} Resource Sharing in the Data Center. In 8th USENIX Symposium on Networked Systems Design and Implementation (NSDI 11). USENIX Association. https:\/\/www.usenix.org\/conference\/nsdi11\/mesos-platform-fine-grained-resource-sharing-data-center"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1142473.1142489"},{"key":"e_1_2_1_36_1","volume-title":"Accessed","year":"2024","unstructured":"Kubernetes. 2024. Kubernetes Overview. https:\/\/kubernetes.io\/docs\/concepts\/overview. Accessed: January 26, 2024."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.5555\/1316689.1316701"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274694.3274720"},{"key":"e_1_2_1_39_1","volume-title":"Accessed","year":"2023","unstructured":"Microsoft. 2023. CLR Integration Code Access Security. https:\/\/learn.microsoft.com\/en-us\/sql\/relational-databases\/clr-integration\/security\/clr-integration-code-access-security. Accessed: February 27, 2024."},{"key":"e_1_2_1_40_1","volume-title":"Accessed","year":"2024","unstructured":"Microsoft. 2024. Azure Blob Storage. https:\/\/azure.microsoft.com\/en-us\/products\/storage\/blobs. Accessed: February 27, 2024."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1287\/isre.2015.0587"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.5555\/1182635.1164217"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.14778\/3476311.3476410"},{"key":"e_1_2_1_44_1","unstructured":"Oracle. 2002. The Virtual Private Database in Oracle9iR2. https:\/\/www.cgisecurity.com\/database\/oracle\/pdf\/VPD9ir2twp.pdf."},{"key":"e_1_2_1_45_1","volume-title":"Accessed","year":"2020","unstructured":"Oracle. 2020. Behind the scenes: How do lambda expressions really work in Java? https:\/\/blogs.oracle.com\/javamagazine\/post\/behind-the-scenes-how-do-lambda-expressions-really-work-in-java. Accessed: January 26, 2024."},{"key":"e_1_2_1_46_1","volume-title":"Accessed","year":"2024","unstructured":"Oracle. 2024. Java Object Serialization. https:\/\/docs.oracle.com\/javase\/8\/docs\/technotes\/guides\/serialization\/index.html. Accessed: January 22, 2024."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.1991.131480"},{"key":"e_1_2_1_48_1","volume-title":"Preventing Privilege Escalation. In 12th USENIX Security Symposium. USENIX Association. https:\/\/www.usenix.org\/conference\/12th-usenix-security-symposium\/preventing-privilege-escalation","author":"Provos Niels","year":"2003","unstructured":"Niels Provos, Markus Friedl, and Peter Honeyman. 2003. Preventing Privilege Escalation. In 12th USENIX Security Symposium. USENIX Association. https:\/\/www.usenix.org\/conference\/12th-usenix-security-symposium\/preventing-privilege-escalation"},{"key":"e_1_2_1_49_1","volume-title":"Accessed","year":"2024","unstructured":"Python. 2024. Python object serialization. https:\/\/docs.python.org\/3\/library\/pickle.html. Accessed: January 26, 2024."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/1007568.1007631"},{"key":"e_1_2_1_51_1","volume-title":"Accessed","year":"2024","unstructured":"Scala. 2024. Case Classes | Tour of Scala | Documentation. https:\/\/docs.scala-lang.org\/tour\/case-classes.html. Accessed: January 26, 2024."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1016\/0169-023X(92)90034-9"},{"key":"e_1_2_1_53_1","volume-title":"Accessed","year":"2020","unstructured":"Snyk. 2020. Serialization and deserialization in Java. https:\/\/snyk.io\/blog\/serialization-and-deserialization-in-java. Accessed: January 26, 2024."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/800182.810400"},{"key":"e_1_2_1_55_1","volume-title":"Accessed","author":"TPC.","year":"2021","unstructured":"TPC. 2021. TPC Benchmark DS. https:\/\/www.tpc.org\/TPC_Documents_Current_Versions\/pdf\/TPC-DS_v3.2.0.pdf. Accessed: February 29, 2024."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2523616.2523633"},{"key":"e_1_2_1_57_1","volume-title":"Building An Elastic Query Engine on Disaggregated Storage. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20)","author":"Vuppalapati Midhul","year":"2020","unstructured":"Midhul Vuppalapati, Justin Miron, Rachit Agarwal, Dan Truong, Ashish Motivala, and Thierry Cruanes. 2020. Building An Elastic Query Engine on Disaggregated Storage. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). USENIX Association, 449--462. https:\/\/www.usenix.org\/conference\/nsdi20\/presentation\/vuppalapati"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3427228.3427640"},{"key":"e_1_2_1_59_1","volume-title":"Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation (NSDI'12)","author":"Zaharia Matei","year":"2012","unstructured":"Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy McCauley, Michael J. Franklin, Scott Shenker, and Ion Stoica. 2012. Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation (NSDI'12). USENIX Association, 15--28."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3685800.3685808","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,31]],"date-time":"2024-12-31T05:33:43Z","timestamp":1735623223000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3685800.3685808"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8]]},"references-count":59,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2024,8]]}},"alternative-id":["10.14778\/3685800.3685808"],"URL":"https:\/\/doi.org\/10.14778\/3685800.3685808","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2024,8]]},"assertion":[{"value":"2024-11-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}