{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,28]],"date-time":"2025-09-28T20:46:36Z","timestamp":1759092396964,"version":"3.41.0"},"reference-count":13,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2015,1,20]],"date-time":"2015-01-20T00:00:00Z","timestamp":1421712000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGOPS Oper. Syst. Rev."],"published-print":{"date-parts":[[2015,1,20]]},"abstract":"<jats:p>This work presents the results of a three-year project that assessed nine different privacy-preserving data search systems. We detail the design of a software assessment framework that focuses on low system footprint, repeatability, and reusability. A unique achievement of this project was the automation and integration of the entire test process, from the production and execution of tests to the generation of human-readable evaluation reports. We synthesize our experiences into a set of simple mantras that we recommend following in the design of any assessment framework.<\/jats:p>","DOI":"10.1145\/2723872.2723877","type":"journal-article","created":{"date-parts":[[2015,1,28]],"date-time":"2015-01-28T14:05:51Z","timestamp":1422453951000},"page":"22-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Automated Assessment of Secure Search Systems"],"prefix":"10.1145","volume":"49","author":[{"given":"Mayank","family":"Varia","sequence":"first","affiliation":[{"name":"MIT Lincoln Laboratory, Lexington, MA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Benjamin","family":"Price","sequence":"additional","affiliation":[{"name":"MIT Lincoln Laboratory, Lexington, MA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nicholas","family":"Hwang","sequence":"additional","affiliation":[{"name":"MIT Lincoln Laboratory, Lexington, MA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ariel","family":"Hamlin","sequence":"additional","affiliation":[{"name":"MIT Lincoln Laboratory, Lexington, MA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonathan","family":"Herzog","sequence":"additional","affiliation":[{"name":"Jonathan Herzog Consulting"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jill","family":"Poland","sequence":"additional","affiliation":[{"name":"MIT Lincoln Laboratory, Lexington, MA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Reschly","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sophia","family":"Yakoubov","sequence":"additional","affiliation":[{"name":"MIT Lincoln Laboratory, Lexington, MA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robert K.","family":"Cunningham","sequence":"additional","affiliation":[{"name":"MIT Lincoln Laboratory, Lexington, MA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2015,1,20]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1002\/spe.4380251203"},{"key":"e_1_2_1_2_1","series-title":"LNCS","first-page":"353","volume-title":"CRYPTO","author":"Cash D.","year":"2013","unstructured":"D. Cash , S. Jarecki , C. S. Jutla , H. Krawczyk , M.-C. Rosu , and M. Steiner . Highly-scalable searchable symmetric encryption with support for boolean queries . In CRYPTO , volume 8042 of LNCS , pages 353 -- 373 . Springer , 2013 . D. Cash, S. Jarecki, C. S. Jutla, H. Krawczyk, M.-C. Rosu, and M. Steiner. Highly-scalable searchable symmetric encryption with support for boolean queries. In CRYPTO, volume 8042 of LNCS, pages 353--373. Springer, 2013."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1327452.1327492"},{"key":"e_1_2_1_4_1","volume-title":"November","author":"Department of Homeland Security.","year":"2009","unstructured":"Department of Homeland Security. A roadmap for cybersecurity research , November 2009 . Department of Homeland Security. A roadmap for cybersecurity research, November 2009."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPEC.2014.7040957"},{"key":"e_1_2_1_6_1","volume-title":"February","author":"IARPA.","year":"2011","unstructured":"IARPA. Broad agency announcement IARPA-BAA-11-01: Security and privacy assurance research (SPAR) program , February 2011 . IARPA. Broad agency announcement IARPA-BAA-11-01: Security and privacy assurance research (SPAR) program, February 2011."},{"key":"e_1_2_1_7_1","first-page":"141","volume-title":"IFIP","author":"Peisert S.","year":"2007","unstructured":"S. Peisert and M. Bishop . How to design computer security experiments . In IFIP , volume 237 , pages 141 -- 148 . Springer , 2007 . S. Peisert and M. Bishop. How to design computer security experiments. In IFIP, volume 237, pages 141--148. Springer, 2007."},{"key":"e_1_2_1_8_1","unstructured":"Project Gutenberg. http:\/\/www.gutenberg.org.  Project Gutenberg. http:\/\/www.gutenberg.org."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2011.155"},{"key":"e_1_2_1_10_1","unstructured":"US Census Bureau. Census 2000 5-percent public use microdata sample (PUMS) files. http:\/\/www2.census.gov\/census_2000\/datasets\/PUMS\/FivePercent\/.  US Census Bureau. Census 2000 5-percent public use microdata sample (PUMS) files. http:\/\/www2.census.gov\/census_2000\/datasets\/PUMS\/FivePercent\/."},{"key":"e_1_2_1_11_1","volume-title":"Genealogy data: Frequently occurring surnames from census","author":"US Census Bureau","year":"2000","unstructured":"US Census Bureau . Genealogy data: Frequently occurring surnames from census 2000 . http:\/\/www. census.gov\/genealogy\/www\/data\/2000surnames\/. US Census Bureau. Genealogy data: Frequently occurring surnames from census 2000. http:\/\/www. census.gov\/genealogy\/www\/data\/2000surnames\/."},{"key":"e_1_2_1_12_1","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1007\/978-3-642-15512-3_12","volume-title":"RAID","author":"Wright C. V.","year":"2010","unstructured":"C. V. Wright , C. Connelly , T. Braje , J. C. Rabek , L. M. Rossey , and R. K. Cunningham . Generating client workloads and high-fidelity network traffic for controllable, repeatable experiments in computer security . In RAID , volume 6307 of Lecture Notes in Computer Science , pages 218 -- 237 . Springer , 2010 . C. V. Wright, C. Connelly, T. Braje, J. C. Rabek, L. M. Rossey, and R. K. Cunningham. Generating client workloads and high-fidelity network traffic for controllable, repeatable experiments in computer security. In RAID, volume 6307 of Lecture Notes in Computer Science, pages 218--237. Springer, 2010."},{"key":"e_1_2_1_13_1","volume-title":"Evaluation of somewhat homomorphic encryption schemes. Master's thesis","author":"Yang Y.","year":"2013","unstructured":"Y. Yang . Evaluation of somewhat homomorphic encryption schemes. Master's thesis , Massachusetts Institute of Technology , 2013 . Y. Yang. Evaluation of somewhat homomorphic encryption schemes. Master's thesis, Massachusetts Institute of Technology, 2013."}],"container-title":["ACM SIGOPS Operating Systems Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2723872.2723877","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2723872.2723877","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T06:16:43Z","timestamp":1750227403000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2723872.2723877"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,1,20]]},"references-count":13,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2015,1,20]]}},"alternative-id":["10.1145\/2723872.2723877"],"URL":"https:\/\/doi.org\/10.1145\/2723872.2723877","relation":{},"ISSN":["0163-5980"],"issn-type":[{"type":"print","value":"0163-5980"}],"subject":[],"published":{"date-parts":[[2015,1,20]]},"assertion":[{"value":"2015-01-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}