{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T07:44:00Z","timestamp":1770536640997,"version":"3.49.0"},"reference-count":27,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2021,7,11]],"date-time":"2021-07-11T00:00:00Z","timestamp":1625961600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGCOMM Comput. Commun. Rev."],"published-print":{"date-parts":[[2021,7,11]]},"abstract":"<jats:p>In January and April 2021 we held the Workshop on Overcoming Measurement Barriers to Internet Research (WOMBIR) with the goal of understanding challenges in network and security data set collection and sharing. Most workshop attendees provided white papers describing their perspectives, and many participated in short-talks and discussion in two virtual workshops over five days. That discussion produced consensus around several points. First, many aspects of the Internet are characterized by decreasing visibility of important network properties, which is in tension with the Internet's role as critical infrastructure. We discussed three specific research areas that illustrate this tension: security, Internet access; and mobile networking. We discussed visibility challenges at all layers of the networking stack, and the challenge of gathering data and validating inferences. Important data sets require longitudinal (long-term, ongoing) data collection and sharing, support for which is more challenging for Internet research than other fields. We discussed why a combination of technical and policy methods are necessary to safeguard privacy when using or sharing measurement data. Workshop participant proposed several opportunities to accelerate progress, some of which require coordination across government, industry, and academia.<\/jats:p>","DOI":"10.1145\/3477482.3477489","type":"journal-article","created":{"date-parts":[[2021,7,23]],"date-time":"2021-07-23T16:04:51Z","timestamp":1627056291000},"page":"33-40","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Workshop on Overcoming Measurement Barriers to Internet Research (WOMBIR 2021) final report"],"prefix":"10.1145","volume":"51","author":[{"given":"KC","family":"Claffy","sequence":"first","affiliation":[{"name":"UC San Diego\/CAIDA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Clark","sequence":"additional","affiliation":[{"name":"MIT CSAIL"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"John","family":"Heidemann","sequence":"additional","affiliation":[{"name":"University of Southern California\/Information Sciences Institute"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fabian","family":"Bustamante","sequence":"additional","affiliation":[{"name":"Northwestern U."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mattijs","family":"Jonker","sequence":"additional","affiliation":[{"name":"University of Twente"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aaron","family":"Schulman","sequence":"additional","affiliation":[{"name":"UC San Diego"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ellen","family":"Zegura","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,7,23]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"M. Martonosi \"Request for Information on the specific needs for datasets to conduct research on computer and network systems.\" NSF Dear Colleague Letter 21-056 at https:\/\/www.nsf.gov\/pubs\/2021\/nsf21056\/nsf21056.jsp.  M. Martonosi \"Request for Information on the specific needs for datasets to conduct research on computer and network systems.\" NSF Dear Colleague Letter 21-056 at https:\/\/www.nsf.gov\/pubs\/2021\/nsf21056\/nsf21056.jsp."},{"key":"e_1_2_1_2_1","volume-title":"Workshop on Overcoming Measurement Barriers to Internet Research (WOMBIR 2021)","year":"2021","unstructured":"\"NSF-sponsored Workshop on Overcoming Measurement Barriers to Internet Research (WOMBIR 2021) -- Parts I and II,\" 2021 . https:\/\/www.caida.org\/workshops\/wombir\/2101\/. \"NSF-sponsored Workshop on Overcoming Measurement Barriers to Internet Research (WOMBIR 2021) -- Parts I and II,\" 2021. https:\/\/www.caida.org\/workshops\/wombir\/2101\/."},{"key":"e_1_2_1_3_1","unstructured":"\"Cyberspace Solarium Commission report \" 2020. https:\/\/www.solarium.gov\/report.  \"Cyberspace Solarium Commission report \" 2020. https:\/\/www.solarium.gov\/report."},{"key":"e_1_2_1_4_1","unstructured":"\"A National Cybersecurity Agenda for Resilient Digital Infrastructure \" The Aspen Institute 2020. https:\/\/www.aspeninstitute.org\/longform\/a-national-cybersecurity-agenda-for-resilient-digital-infrastructure\/.  \"A National Cybersecurity Agenda for Resilient Digital Infrastructure \" The Aspen Institute 2020. https:\/\/www.aspeninstitute.org\/longform\/a-national-cybersecurity-agenda-for-resilient-digital-infrastructure\/."},{"key":"e_1_2_1_5_1","unstructured":"National Academy of Sciences \"Principles and Practices for a Federal Statistical Agency Edition 7.\" https:\/\/www.nap.edu\/resource\/25885\/P&P%207%20Higlights.pdf.  National Academy of Sciences \"Principles and Practices for a Federal Statistical Agency Edition 7.\" https:\/\/www.nap.edu\/resource\/25885\/P&P%207%20Higlights.pdf."},{"key":"e_1_2_1_6_1","unstructured":"\"Telecommunications Act of 1996 Section 706(a): Advanced Telecommunications Incentives.\" (codified as 47 U.S.C. 1302).  \"Telecommunications Act of 1996 Section 706(a): Advanced Telecommunications Incentives.\" (codified as 47 U.S.C. 1302)."},{"key":"e_1_2_1_7_1","unstructured":"U.S. General Services Administration (GSA) \"Broadband Data Resources in data.gov \" 2021. https:\/\/catalog.data.gov\/dataset?q=broadband.  U.S. General Services Administration (GSA) \"Broadband Data Resources in data.gov \" 2021. https:\/\/catalog.data.gov\/dataset?q=broadband."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2663716.2663753"},{"key":"e_1_2_1_9_1","volume-title":"C2B(1)D '15, (New York","author":"Rula J. P.","year":"2015","unstructured":"J. P. Rula , Z. S. Bischof , and F. E. Bustamante , \" Second chance: Understanding diversity in broadband access network performance,\" in Proceedings of the 2015 ACM SIGCOMM Workshop on Crowdsourcing and Crowdsharing of Big (Internet) Data , C2B(1)D '15, (New York , NY , USA), p. 914, Association for Computing Machinery , 2015 . J. P. Rula, Z. S. Bischof, and F. E. Bustamante, \"Second chance: Understanding diversity in broadband access network performance,\" in Proceedings of the 2015 ACM SIGCOMM Workshop on Crowdsourcing and Crowdsharing of Big (Internet) Data, C2B(1)D '15, (New York, NY, USA), p. 914, Association for Computing Machinery, 2015."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-54328-4_12"},{"key":"e_1_2_1_11_1","volume-title":"MobileInsight: Extracting and Analyzing Cellular Network Information on Smartphones.\" http:\/\/www.mobileinsight.net","author":"Li Y.","year":"2016","unstructured":"Y. Li , C. Peng , Z. Yuan , J. Li , H. Deng , and T. Wang , \" MobileInsight: Extracting and Analyzing Cellular Network Information on Smartphones.\" http:\/\/www.mobileinsight.net , 2016 . Y. Li, C. Peng, Z. Yuan, J. Li, H. Deng, and T. Wang, \"MobileInsight: Extracting and Analyzing Cellular Network Information on Smartphones.\" http:\/\/www.mobileinsight.net, 2016."},{"key":"e_1_2_1_12_1","volume-title":"Ethical Principles Guiding Information and Communication Technology Research","author":"Dittrich Erin","year":"2012","unstructured":"Kenneally, Erin and Dittrich , David, \" The Menlo Report : Ethical Principles Guiding Information and Communication Technology Research ,\" 2012 . http:\/\/ssrn.com\/abstract=2445102. Kenneally, Erin and Dittrich, David, \"The Menlo Report: Ethical Principles Guiding Information and Communication Technology Research,\" 2012. http:\/\/ssrn.com\/abstract=2445102."},{"key":"e_1_2_1_13_1","volume-title":"A Companion to the Menlo Report","author":"Kenneally David","year":"2013","unstructured":"Dittrich, David and Kenneally , Erin and Bailey , Michael, \" Applying Ethical Principles to Information and Communication Technology Research : A Companion to the Menlo Report ,\" 2013 . http:\/\/ssrn.com\/abstract=2342036. Dittrich, David and Kenneally, Erin and Bailey, Michael, \"Applying Ethical Principles to Information and Communication Technology Research: A Companion to the Menlo Report,\" 2013. http:\/\/ssrn.com\/abstract=2342036."},{"key":"e_1_2_1_14_1","volume-title":"Calibrating noise to sensitivity in private data analysis,\" in Theory of Cryptography","author":"Dwork C.","year":"2006","unstructured":"C. Dwork , F. McSherry , K. Nissim , and A. Smith , \" Calibrating noise to sensitivity in private data analysis,\" in Theory of Cryptography (S. Halevi and T. Rabin, eds.), 2006 . C. Dwork, F. McSherry, K. Nissim, and A. Smith, \"Calibrating noise to sensitivity in private data analysis,\" in Theory of Cryptography (S. Halevi and T. Rabin, eds.), 2006."},{"key":"e_1_2_1_15_1","volume-title":"ACM","volume":"64","author":"Staff C.","year":"2021","unstructured":"C. Staff , \" Differential privacy : The pursuit of protections by default,\" Commun . ACM , vol. 64 , Jan. 2021 . C. Staff, \"Differential privacy: The pursuit of protections by default,\" Commun. ACM, vol. 64, Jan. 2021."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1561\/9781680835090"},{"key":"e_1_2_1_17_1","doi-asserted-by":"crossref","unstructured":"Y. Lindell \"Secure multiparty computation \" Comm. ACM Dec. 2020.  Y. Lindell \"Secure multiparty computation \" Comm. ACM Dec. 2020.","DOI":"10.1145\/3387108"},{"key":"e_1_2_1_18_1","first-page":"141","author":"Francis P.","year":"2017","unstructured":"P. Francis , S. Eide , and R. Munz , \"Diffix: High-utility database anonymization,\" in Annual Privacy Forum , pp. 141 -- 158 , 06 2017 . P. Francis, S. Eide, and R. Munz, \"Diffix: High-utility database anonymization,\" in Annual Privacy Forum, pp. 141--158, 06 2017.","journal-title":"\"Diffix: High-utility database anonymization,\" in Annual Privacy Forum"},{"key":"e_1_2_1_19_1","volume-title":"NSDI'17","author":"Corrigan-Gibbs H.","year":"2017","unstructured":"H. Corrigan-Gibbs and D. Boneh , \" Prio: Private, robust, and scalable computation of aggregate statistics,\" in USENIX Conference on Networked Systems Design and Implementation , NSDI'17 , 2017 . H. Corrigan-Gibbs and D. Boneh, \"Prio: Private, robust, and scalable computation of aggregate statistics,\" in USENIX Conference on Networked Systems Design and Implementation, NSDI'17, 2017."},{"key":"e_1_2_1_20_1","first-page":"593628","volume-title":"Shades of gray: Seeing the full spectrum of practical data de-identification,\" Santa Clara Law Review","author":"Polonetsky J.","year":"2016","unstructured":"J. Polonetsky , O. Tene , and K. Finch , \" Shades of gray: Seeing the full spectrum of practical data de-identification,\" Santa Clara Law Review , p. 593628 , 2016 . http:\/\/digitalcommons.law.scu.edu\/lawreview\/vol56\/iss3\/3. J. Polonetsky, O. Tene, and K. Finch, \"Shades of gray: Seeing the full spectrum of practical data de-identification,\" Santa Clara Law Review, p. 593628, 2016. http:\/\/digitalcommons.law.scu.edu\/lawreview\/vol56\/iss3\/3."},{"key":"e_1_2_1_21_1","unstructured":"\"CAIDA Data Stewardship Agreements \" 2020. https:\/\/www.caida.org\/about\/legal\/.  \"CAIDA Data Stewardship Agreements \" 2020. https:\/\/www.caida.org\/about\/legal\/."},{"key":"e_1_2_1_22_1","unstructured":"kc claffy and David Clark \"Comments on the American Research Environment \" 2021. in response to Request for Information from the National Science and Technology Council https:\/\/www.caida.org\/catalog\/papers\/2020_comments_rfi_american_research\/.  kc claffy and David Clark \"Comments on the American Research Environment \" 2021. in response to Request for Information from the National Science and Technology Council https:\/\/www.caida.org\/catalog\/papers\/2020_comments_rfi_american_research\/."},{"key":"e_1_2_1_23_1","unstructured":"University Industry Demonstration Partnership \"Uidp website.\" https:\/\/uidp.org\/ 2021.  University Industry Demonstration Partnership \"Uidp website.\" https:\/\/uidp.org\/ 2021."},{"key":"e_1_2_1_24_1","volume-title":"Jan.","author":"NSF","year":"2021","unstructured":"NSF , NTIA, and FCC , \"Memorandum of agreement between the NSF , the NTIA and the FCC regarding the Spectrum Innovation Initiative.\" https:\/\/www.ntia.doc.gov\/files\/ntia\/blogimages\/sii_moa_fcc_nsf_ntia.pdf , Jan. 2021 . NSF, NTIA, and FCC, \"Memorandum of agreement between the NSF, the NTIA and the FCC regarding the Spectrum Innovation Initiative.\" https:\/\/www.ntia.doc.gov\/files\/ntia\/blogimages\/sii_moa_fcc_nsf_ntia.pdf, Jan. 2021."},{"key":"e_1_2_1_25_1","unstructured":"FCC \"Electronic comment filing system.\" https:\/\/www.fcc.gov\/ecfs 2021.  FCC \"Electronic comment filing system.\" https:\/\/www.fcc.gov\/ecfs 2021."},{"key":"e_1_2_1_26_1","unstructured":"FCC. https:\/\/www.fcc.gov\/proceedings-actions\/daily-digest.  FCC. https:\/\/www.fcc.gov\/proceedings-actions\/daily-digest."},{"key":"e_1_2_1_27_1","unstructured":"NFS and FCC \"NSF-FCC workshop on tracking quality of experience in the Internet.\" Workshop website http:\/\/aqualab.cs.northwestern.edu\/conference\/276 Oct. 2015.  NFS and FCC \"NSF-FCC workshop on tracking quality of experience in the Internet.\" Workshop website http:\/\/aqualab.cs.northwestern.edu\/conference\/276 Oct. 2015."}],"container-title":["ACM SIGCOMM Computer Communication Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3477482.3477489","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3477482.3477489","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:32Z","timestamp":1750188632000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3477482.3477489"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,11]]},"references-count":27,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,7,11]]}},"alternative-id":["10.1145\/3477482.3477489"],"URL":"https:\/\/doi.org\/10.1145\/3477482.3477489","relation":{},"ISSN":["0146-4833"],"issn-type":[{"value":"0146-4833","type":"print"}],"subject":[],"published":{"date-parts":[[2021,7,11]]},"assertion":[{"value":"2021-07-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}