{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,1]],"date-time":"2026-01-01T10:06:30Z","timestamp":1767261990601,"version":"3.41.0"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2020,5,22]],"date-time":"2020-05-22T00:00:00Z","timestamp":1590105600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGCOMM Comput. Commun. Rev."],"published-print":{"date-parts":[[2020,5,22]]},"abstract":"<jats:p>A commonly held belief is that traffic engineering and routing changes are infrequent. However, based on our measurements over a number of years of traffic between data centers in one of the largest cloud provider's networks, we found that it is common for flows to change paths at ten-second intervals or even faster. These frequent path and, consequently, latency variations can negatively impact the performance of cloud applications, specifically, latency-sensitive and geo-distributed applications.<\/jats:p>\n          <jats:p>Our recent measurements and analysis focused on observing path changes and latency variations between different Amazon aws regions. To this end, we devised a path change detector that we validated using both ad hoc experiments and feedback from cloud networking experts. The results provide three main insights: (1) Traffic Engineering (TE) frequently moves (TCP and UDP) flows among network paths of different latency, (2) Flows experience unfair performance, where a subset of flows between two machines can suffer large latency penalties (up to 32% at the 95th percentile) or excessive number of latency changes, and (3) Tenants may have incentives to selfishly move traffic to low latency classes (to boost the performance of their applications). We showcase this third insight with an example using rsync synchronization.<\/jats:p>\n          <jats:p>To the best of our knowledge, this is the first paper to reveal the high frequency of TE activity within a large cloud provider's network. Based on these observations, we expect our paper to spur discussions and future research on how cloud providers and their tenants can ultimately reconcile their independent and possibly conflicting objectives. Our data is publicly available for reproducibility and further analysis at http:\/\/goo.gl\/25BKte.<\/jats:p>","DOI":"10.1145\/3402413.3402416","type":"journal-article","created":{"date-parts":[[2020,5,25]],"date-time":"2020-05-25T22:41:33Z","timestamp":1590446493000},"page":"11-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":21,"title":["Path persistence in the cloud"],"prefix":"10.1145","volume":"50","author":[{"given":"Waleed","family":"Reda","sequence":"first","affiliation":[{"name":"KTH Royal Institute of Technology \/ Universite catholique de Louvain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kirill","family":"Bogdanov","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexandros","family":"Milolidakis","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hamid","family":"Ghasemirahni","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marco","family":"Chiesa","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"suffix":"Jr.","given":"Gerald Q.","family":"Maguire","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dejan","family":"Kosti\u0107","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,5,23]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"All measurement data used in this paper can be found via this link goo.gl\/25BKte  All measurement data used in this paper can be found via this link goo.gl\/25BKte"},{"key":"e_1_2_1_2_1","volume-title":"Balakrishnan","author":"Arun V.","year":"2018","unstructured":"Arun , V. , Balakrishnan , H. : Copa: Practical delay-based congestion control for the internet. In: 15th ${$USENIX$}$ Symposium on Networked Systems Design and Implementation (${$NSDI$}$ 18). USENIX Association ( 2018 ) Arun, V., Balakrishnan, H.: Copa: Practical delay-based congestion control for the internet. In: 15th ${$USENIX$}$ Symposium on Networked Systems Design and Implementation (${$NSDI$}$ 18). USENIX Association (2018)"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1177080.1177100"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1298306.1298329"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/E2EMON.2007.375313"},{"key":"e_1_2_1_6_1","unstructured":"AWS A.: Setting the Time for Your Linux Instance https:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/set-time.html Accessed 2-Feb-2019  AWS A.: Setting the Time for Your Linux Instance https:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/set-time.html Accessed 2-Feb-2019"},{"key":"e_1_2_1_7_1","unstructured":"Bing: Microsoft Bing https:\/\/www.bing.com Accessed 1-Oct-2018  Bing: Microsoft Bing https:\/\/www.bing.com Accessed 1-Oct-2018"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/510726.510728"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2806777.2806939"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3267809.3267820"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1282380.1282382"},{"key":"e_1_2_1_12_1","first-page":"235","volume-title":"Diot","author":"Cunha","year":"2011","unstructured":"Cunha , \u00cd., Teixeira , R. , Diot , C. : Measuring and characterizing end-to-end route dynamics in the presence of load balancing. In : PAM. vol. 11, pp. 235 -- 244 . Springer ( 2011 ) Cunha, \u00cd., Teixeira, R., Diot, C.: Measuring and characterizing end-to-end route dynamics in the presence of load balancing. In: PAM. vol. 11, pp. 235--244. Springer (2011)"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2043164.2018451"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.comnet.2012.12.011"},{"key":"e_1_2_1_15_1","volume-title":"Gandhi","author":"Dhody D.","year":"2018","unstructured":"Dhody , D. , Palle , U. , Singh , R. , Gandhi , R. : PCEP Extensions for MPLS-TE LSP Automatic Bandwidth Adjustment with Stateful PCE. Internet Drafts PCE Working Group (Nov 2018 ), https:\/\/tools.ietf.org\/html\/draft-ietf-pce-stateful-pce-auto-bandwidth-08 Dhody, D., Palle, U., Singh, R., Gandhi, R.: PCEP Extensions for MPLS-TE LSP Automatic Bandwidth Adjustment with Stateful PCE. Internet Drafts PCE Working Group (Nov 2018), https:\/\/tools.ietf.org\/html\/draft-ietf-pce-stateful-pce-auto-bandwidth-08"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2002.1003042"},{"key":"e_1_2_1_17_1","unstructured":"Hamilton J.: Aws re:invent 2016: Tuesday night live https:\/\/www.youtube.com\/watch?v=AyOAjFNPAbA Accessed 17-Jun-2018  Hamilton J.: Aws re:invent 2016: Tuesday night live https:\/\/www.youtube.com\/watch?v=AyOAjFNPAbA Accessed 17-Jun-2018"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.17487\/RFC1058"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2716281.2836113"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2534169.2486012"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.17487\/RFC2992"},{"key":"e_1_2_1_22_1","doi-asserted-by":"crossref","unstructured":"Jain S. Kumar A. Mandal S. Ong J. Poutievski L. Singh A. Venkata S. Wanderer J. Zhou J. Zhu M. Zolla J. H\u00f6lzle U. Stuart S. Vahdat A.: B4: Experience with a Globally-Deployed Software Defined WAN. In: SIGCOMM (2013)  Jain S. Kumar A. Mandal S. Ong J. Poutievski L. Singh A. Venkata S. Wanderer J. Zhou J. Zhu M. Zolla J. H\u00f6lzle U. Stuart S. Vahdat A.: B4: Experience with a Globally-Deployed Software Defined WAN. In: SIGCOMM (2013)","DOI":"10.1145\/2486001.2486019"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2934872.2934893"},{"key":"e_1_2_1_24_1","volume-title":"Gendel","author":"Laor M.","year":"2002","unstructured":"Laor , M. , Gendel , L. : The effect of packet reordering in a backbone link on application throughput. IEEE network 16(5), 28--36 ( 2002 ) Laor, M., Gendel, L.: The effect of packet reordering in a backbone link on application throughput. IEEE network 16(5), 28--36 (2002)"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/505688.505692"},{"key":"e_1_2_1_26_1","first-page":"367","volume-title":"Proceedings of the 7th symposium on Operating systems design and implementation.","author":"Madhyastha H.V.","year":"2006","unstructured":"Madhyastha , H.V. , Isdal , T. , Piatek , M. , Dixon , C. , Anderson , T. , Krishnamurthy , A. , Venkataramani , A. : iPlane: An Information Plane for Distributed Services . In: Proceedings of the 7th symposium on Operating systems design and implementation. pp. 367 -- 380 . USENIX Association ( 2006 ) Madhyastha, H.V., Isdal, T., Piatek, M., Dixon, C., Anderson, T., Krishnamurthy, A., Venkataramani, A.: iPlane: An Information Plane for Distributed Services. In: Proceedings of the 7th symposium on Operating systems design and implementation. pp. 367--380. USENIX Association (2006)"},{"key":"e_1_2_1_27_1","unstructured":"Meyer D.: AWS Remains Dominant Player in Growing Cloud Market SRG Reports https:\/\/www.sdxcentral.com\/articles\/news\/aws-remains-dominant-player-in-growing-cloud-market-srg-reports\/  Meyer D.: AWS Remains Dominant Player in Growing Cloud Market SRG Reports https:\/\/www.sdxcentral.com\/articles\/news\/aws-remains-dominant-player-in-growing-cloud-market-srg-reports\/"},{"key":"e_1_2_1_28_1","volume-title":"OSPF Version 2. Internet Request for Comments RFC 2328 (INTERNET STANDARD) (Apr","author":"Moy J.","year":"1998","unstructured":"Moy , J. : OSPF Version 2. Internet Request for Comments RFC 2328 (INTERNET STANDARD) (Apr 1998 ), http:\/\/www.rfc-editor.org\/rfc\/rfc2328.txt, updated by RFCs 5709, 6549, 6845, 6860, 7474 Moy, J.: OSPF Version 2. Internet Request for Comments RFC 2328 (INTERNET STANDARD) (Apr 1998), http:\/\/www.rfc-editor.org\/rfc\/rfc2328.txt, updated by RFCs 5709, 6549, 6845, 6860, 7474"},{"key":"e_1_2_1_29_1","volume-title":"Maltz","author":"Pathak A.","year":"2011","unstructured":"Pathak , A. , Zhang , M. , Hu , Y.C. , Mahajan , R. , Maltz , D.A. : Latency inflation with MPLS-based traffic engineering. In : IMC ( 2011 ) Pathak, A., Zhang, M., Hu, Y.C., Mahajan, R., Maltz, D.A.: Latency inflation with MPLS-based traffic engineering. In: IMC (2011)"},{"key":"e_1_2_1_30_1","volume-title":"Bush","author":"Pelsser C.","year":"2013","unstructured":"Pelsser , C. , Cittadini , L. , Vissicchio , S. , Bush , R. : From Paris to Tokyo : on the suitability of ping to measure latency. In : IMC ( 2013 ) Pelsser, C., Cittadini, L., Vissicchio, S., Bush, R.: From Paris to Tokyo: on the suitability of ping to measure latency. In: IMC (2013)"},{"issue":"1","key":"e_1_2_1_31_1","first-page":"73","volume":"35","author":"Pucha H.","year":"2007","unstructured":"Pucha , H. , Zhang , Y. , Mao , Z.M. , Hu , Y.C. : Understanding Network Delay Changes Caused by Routing Events. ACM SIGMETRICS Performance Evaluation Review - SIGMETRICS '07 Conference Proceedings 35 ( 1 ), 73 -- 84 ( 2007 ) Pucha, H., Zhang, Y., Mao, Z.M., Hu, Y.C.: Understanding Network Delay Changes Caused by Routing Events. ACM SIGMETRICS Performance Evaluation Review - SIGMETRICS '07 Conference Proceedings 35(1), 73--84 (2007)","journal-title":"Conference Proceedings"},{"key":"e_1_2_1_32_1","volume-title":"Callon","author":"Rosen E.","year":"2001","unstructured":"Rosen , E. , Viswanathan , A. , Callon , R. : Multiprotocol Label Switching Architecture. Internet Request for Comments RFC 3031 (Standards Track) (Jan 2001 ), http:\/\/www.rfc-editor.org\/rfc\/rfc3031.txt Rosen, E., Viswanathan, A., Callon, R.: Multiprotocol Label Switching Architecture. Internet Request for Comments RFC 3031 (Standards Track) (Jan 2001), http:\/\/www.rfc-editor.org\/rfc\/rfc3031.txt"},{"key":"e_1_2_1_33_1","doi-asserted-by":"crossref","unstructured":"Roughgarden T. Tardos \u00c9.: How bad is selfish routing? Journal of the ACM (JACM) 49(2) 236--259 (2002)  Roughgarden T. Tardos \u00c9.: How bad is selfish routing? Journal of the ACM (JACM) 49 (2) 236--259 (2002)","DOI":"10.1145\/506147.506153"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2785956.2787472"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/GLOCOM.2004.1378050"},{"key":"e_1_2_1_36_1","first-page":"407","volume-title":"14th USENIX Symposium on Networked Systems Design and Implementation NSDI'17)","author":"Vanini E.","year":"2017","unstructured":"Vanini , E. , Pan , R. , Alizadeh , M. , Taheri , P. , Edsall , T. : Let it flow: Resilient asymmetric load balancing with flowlet switching . In: 14th USENIX Symposium on Networked Systems Design and Implementation NSDI'17) . pp. 407 -- 420 ( 2017 ) Vanini, E., Pan, R., Alizadeh, M., Taheri, P., Edsall, T.: Let it flow: Resilient asymmetric load balancing with flowlet switching. In: 14th USENIX Symposium on Networked Systems Design and Implementation NSDI'17). pp. 407--420 (2017)"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2213836.2213957"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICNP.2003.1249760"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/INFCOM.2000.832203"}],"container-title":["ACM SIGCOMM Computer Communication Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3402413.3402416","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3402413.3402416","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:41:34Z","timestamp":1750200094000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3402413.3402416"}},"subtitle":["A study of the effects of inter-region traffic engineering in a large cloud provider's network"],"short-title":[],"issued":{"date-parts":[[2020,5,22]]},"references-count":39,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2020,5,22]]}},"alternative-id":["10.1145\/3402413.3402416"],"URL":"https:\/\/doi.org\/10.1145\/3402413.3402416","relation":{},"ISSN":["0146-4833"],"issn-type":[{"type":"print","value":"0146-4833"}],"subject":[],"published":{"date-parts":[[2020,5,22]]},"assertion":[{"value":"2020-05-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}