{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:30:22Z","timestamp":1750307422207,"version":"3.41.0"},"reference-count":26,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2010,4,9]],"date-time":"2010-04-09T00:00:00Z","timestamp":1270771200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGCOMM Comput. Commun. Rev."],"published-print":{"date-parts":[[2010,4,9]]},"abstract":"<jats:p>Today's open platforms for network measurement and distributed system research, which we collectively refer to as testbeds in this article, provide opportunities for controllable experimentation and evaluations of systems at the scale of hundreds or thousands of hosts. In this article, we identify several issues with extending results from such platforms to Internet wide perspectives. Specifically, we try to quantify the level of inaccuracy and incompleteness of testbed results when applied to the context of a large-scale peer-to-peer (P2P) system. Based on our results, we emphasize the importance of measurements in the appropriate environment when evaluating Internet-scale systems.<\/jats:p>","DOI":"10.1145\/1764873.1764880","type":"journal-article","created":{"date-parts":[[2010,4,12]],"date-time":"2010-04-12T18:13:20Z","timestamp":1271096000000},"page":"43-50","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Pitfalls for testbed evaluations of internet systems"],"prefix":"10.1145","volume":"40","author":[{"given":"David R.","family":"Choffnes","sequence":"first","affiliation":[{"name":"Northwestern University, Evanston, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fabian E.","family":"Bustamante","sequence":"additional","affiliation":[{"name":"Northwestern University, Evanston, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2010,4,9]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1592568.1592605"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1644893.1644923"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1658939.1658964"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1402958.1403000"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.5555\/1833515.1833574"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1658997.1659002"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015467.1015471"},{"key":"e_1_2_1_8_1","unstructured":"EMULAB. Emulab. http:\/\/www.emulab.net\/.  EMULAB. Emulab. http:\/\/www.emulab.net\/."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/637201.637203"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/1762888.1762894"},{"key":"e_1_2_1_11_1","volume-title":"Proc. of USENIX NSDI (Apr.","author":"LEDLIE J.","year":"2007","unstructured":"LEDLIE , J. , GARDNER , P. , AND SELTZER , M. Network coordinates in the wild . In Proc. of USENIX NSDI (Apr. 2007 ). LEDLIE, J., GARDNER, P., AND SELTZER, M. Network coordinates in the wild. In Proc. of USENIX NSDI (Apr. 2007)."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1644893.1644914"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/INFCOM.2002.1019258"},{"key":"e_1_2_1_14_1","volume-title":"Proc. of HotNets","author":"PIATEK M.","year":"2009","unstructured":"PIATEK , M. , MADHYASTHA , H. V. , JOHN , J. P. , KRISHNAMURTHY , A. , AND ANDERSON , T. Pitfalls for ISP-friendly P2P design . In Proc. of HotNets ( 2009 ). PIATEK, M., MADHYASTHA, H. V., JOHN, J. P., KRISHNAMURTHY, A., AND ANDERSON, T. Pitfalls for ISP-friendly P2P design. In Proc. of HotNets (2009)."},{"key":"e_1_2_1_15_1","unstructured":"PLANETLAB. Planetlab. http:\/\/www.planet-lab.org\/.  PLANETLAB. Planetlab. http:\/\/www.planet-lab.org\/."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/MNET.2003.1248658"},{"key":"e_1_2_1_17_1","volume-title":"Proc. of IEEE INFOCOM (Apr.","author":"RABINOVICH M.","year":"2006","unstructured":"RABINOVICH , M. , TRIUKOSE , S. , WEN , Z. , AND WANG , L. Dipzoom : The internet measurements marketplace . In Proc. of IEEE INFOCOM (Apr. 2006 ). RABINOVICH, M., TRIUKOSE, S., WEN, Z., AND WANG, L. Dipzoom: The internet measurements marketplace. In Proc. of IEEE INFOCOM (Apr. 2006)."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCCN.2009.5235346"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1452520.1452558"},{"key":"e_1_2_1_20_1","unstructured":"ROUTEVIEWS. http:\/\/www.routeviews.org\/.  ROUTEVIEWS. http:\/\/www.routeviews.org\/."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2008.54"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/948205.948223"},{"key":"e_1_2_1_23_1","unstructured":"TEAM CYMRU. The Team Cymru IP to ASN lookup page. http:\/\/www.cymru.com\/BGP\/asnlookup.html.  TEAM CYMRU. The Team Cymru IP to ASN lookup page. http:\/\/www.cymru.com\/BGP\/asnlookup.html."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1298306.1298331"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1080091.1080103"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1402958.1402999"}],"container-title":["ACM SIGCOMM Computer Communication Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1764873.1764880","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1764873.1764880","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T11:39:44Z","timestamp":1750246784000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1764873.1764880"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2010,4,9]]},"references-count":26,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2010,4,9]]}},"alternative-id":["10.1145\/1764873.1764880"],"URL":"https:\/\/doi.org\/10.1145\/1764873.1764880","relation":{},"ISSN":["0146-4833"],"issn-type":[{"type":"print","value":"0146-4833"}],"subject":[],"published":{"date-parts":[[2010,4,9]]},"assertion":[{"value":"2010-04-09","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}