{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,27]],"date-time":"2026-09-27T14:50:30Z","timestamp":1790520630894,"version":"4.1.0"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2013,8,1]],"date-time":"2013-08-01T00:00:00Z","timestamp":1375315200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["0546173, 0720665, 0721680, 0831647, 0855141, 1017898, 1054788, 1219004"],"award-info":[{"award-number":["0546173, 0720665, 0721680, 0831647, 0855141, 1017898, 1054788, 1219004"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Syst."],"published-print":{"date-parts":[[2013,8]]},"abstract":"<jats:p>Virtualization is a key technology that powers cloud computing platforms such as Amazon EC2. Virtual machine (VM) consolidation, where multiple VMs share a physical host, has seen rapid adoption in practice, with increasingly large numbers of VMs per machine and per CPU core. Our investigations, however, suggest that the increasing degree of VM consolidation has serious negative effects on the VMs\u2019 TCP performance. As multiple VMs share a given CPU, the scheduling latencies, which can be in the order of tens of milliseconds, substantially increase the typically submillisecond round-trip times (RTTs) for TCP connections in a datacenter, causing significant degradation in throughput. In this article, we propose a lightweight solution, called vPRO, that (a) offloads the VM\u2019s TCP congestion control function to the driver domain to improve TCP transmit performance; and (b) offloads TCP acknowledgment functionality to the driver domain to improve the TCP receive performance. Our evaluation of a vPRO prototype on Xen suggests that vPRO substantially improves TCP receive and transmit throughputs with minimal per-packet CPU overhead. We further show that the higher TCP throughput leads to improvement in application-level performance, via experiments with Apache Olio, a Web 2.0 cloud application, and Intel MPI benchmark.<\/jats:p>","DOI":"10.1145\/2491463","type":"journal-article","created":{"date-parts":[[2020,4,4]],"date-time":"2020-04-04T07:41:33Z","timestamp":1585986093000},"page":"1-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":18,"title":["Protocol Responsibility Offloading to Improve TCP Throughput in Virtualized Environments"],"prefix":"10.1145","volume":"31","author":[{"given":"Sahan","family":"Gamage","sequence":"first","affiliation":[{"name":"Purdue University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ramana Rao","family":"Kompella","sequence":"additional","affiliation":[{"name":"Purdue University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dongyan","family":"Xu","sequence":"additional","affiliation":[{"name":"Purdue University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ardalan","family":"Kangarlou","sequence":"additional","affiliation":[{"name":"NetApp Inc. and Purdue University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2013,8]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1851182.1851192"},{"key":"e_1_2_1_2_1","unstructured":"Amazon EC2. 2013. Amazon EC2 instance types. http:\/\/aws.amazon.com\/ec2\/instance-types\/.  Amazon EC2. 2013. Amazon EC2 instance types. http:\/\/aws.amazon.com\/ec2\/instance-types\/."},{"key":"e_1_2_1_3_1","volume-title":"EECS Dept.","author":"Armbrust Michael","unstructured":"Michael Armbrust , Armando Fox , Rean Griffith , Anthony D. Joseph , Randy Katz , Andy Konwinski , Gunho Lee , David A. Patterson , Ariel Rabkin , Ion Stoica , and Matei Zaharia . 2009. Above the Clouds: A Berkeley view of cloud computing. Tech. rep. UCB\/EECS-2009-28 , EECS Dept. , University of California , Berkeley. Michael Armbrust, Armando Fox, Rean Griffith, Anthony D. Joseph, Randy Katz, Andy Konwinski, Gunho Lee, David A. Patterson, Ariel Rabkin, Ion Stoica, and Matei Zaharia. 2009. Above the Clouds: A Berkeley view of cloud computing. Tech. rep. UCB\/EECS-2009-28, EECS Dept., University of California, Berkeley."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/215530.215544"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1672308.1672325"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/49.464716"},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the Conference on USENIX Annual Technical Conference (ATC\u201909)","author":"Burtsev Anton","unstructured":"Anton Burtsev , Kiran Srinivasan , Prashanth Radhakrishnan , Lakshmi N. Bairavasundaram , Kaladhar Vornganti , and Garth R. Goodson . 2009. Fido: Fast inter-virtual-machine communication for enterprise appliances . In Proceedings of the Conference on USENIX Annual Technical Conference (ATC\u201909) . 25--25. Anton Burtsev, Kiran Srinivasan, Prashanth Radhakrishnan, Lakshmi N. Bairavasundaram, Kaladhar Vornganti, and Garth R. Goodson. 2009. Fido: Fast inter-virtual-machine communication for enterprise appliances. In Proceedings of the Conference on USENIX Annual Technical Conference (ATC\u201909). 25--25."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/90.664262"},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the 6th Conference on Symposium on Operating Systems Design & Implementation (OSDI&rsquo;\u201904)","author":"Dean J.","unstructured":"J. Dean and S. Ghemawat . 2004. MapReduce: Simplified data processing on large clusters . In Proceedings of the 6th Conference on Symposium on Operating Systems Design & Implementation (OSDI&rsquo;\u201904) . 10--10. J. Dean and S. Ghemawat. 2004. MapReduce: Simplified data processing on large clusters. In Proceedings of the 6th Conference on Symposium on Operating Systems Design & Implementation (OSDI&rsquo;\u201904). 10--10."},{"key":"e_1_2_1_10_1","volume-title":"Proceedings of the 1st Conference on I\/O Virtualization (WIOV\u201908)","author":"Dong Yaozu","year":"2008","unstructured":"Yaozu Dong , Zhao Yu , and Greg Rose . 2008 . SR-IOV networking in Xen: Architecture, design and implementation . In Proceedings of the 1st Conference on I\/O Virtualization (WIOV\u201908) . 10--10. Yaozu Dong, Zhao Yu, and Greg Rose. 2008. SR-IOV networking in Xen: Architecture, design and implementation. In Proceedings of the 1st Conference on I\/O Virtualization (WIOV\u201908). 10--10."},{"key":"e_1_2_1_11_1","unstructured":"FABAN. 2013. Project Faban. https:\/\/java.net\/projects\/faban\/.  FABAN. 2013. Project Faban. https:\/\/java.net\/projects\/faban\/."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2038916.2038940"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1254810.1254828"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1592568.1592576"},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201910)","author":"Gulati Ajay","year":"2010","unstructured":"Ajay Gulati , Arif Merchant , and Peter Varman . 2010 . mClock: Handling throughput variability for hypervisor IO scheduling . In Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201910) . 1--7. Ajay Gulati, Arif Merchant, and Peter Varman. 2010. mClock: Handling throughput variability for hypervisor IO scheduling. In Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201910). 1--7."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.5555\/1515984.1516011"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201908)","author":"Gupta Diwaker","year":"2008","unstructured":"Diwaker Gupta , Sangmin Lee , Michael Vrable , Stefan Savage , Alex C. Snoeren , George Varghese , Geoffrey M. Voelker , and Amin Vahdat . 2008 . Difference engine: Harnessing memory redundancy in virtual machines . In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201908) . 309--322. Diwaker Gupta, Sangmin Lee, Michael Vrable, Stefan Savage, Alex C. Snoeren, George Varghese, Geoffrey M. Voelker, and Amin Vahdat. 2008. Difference engine: Harnessing memory redundancy in virtual machines. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201908). 309--322."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1400097.1400105"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1362622.1362635"},{"key":"e_1_2_1_20_1","unstructured":"IMB. 2011. Intel MPI benchmark. http:\/\/software.intel.com\/en-us\/articles\/intel-mpi-benchmarks\/.  IMB. 2011. Intel MPI benchmark. http:\/\/software.intel.com\/en-us\/articles\/intel-mpi-benchmarks\/."},{"key":"e_1_2_1_21_1","unstructured":"IPERF. 2013. The Iperf benchmark. https:\/\/code.google.com\/p\/iperf\/.  IPERF. 2013. The Iperf benchmark. https:\/\/code.google.com\/p\/iperf\/."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1644893.1644918"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1996130.1996164"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2010.57"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807128.1807135"},{"key":"e_1_2_1_26_1","volume-title":"Proceedings of the 2nd Workshop on I\/O Virtualization (WIOV\u201910)","author":"Kesavan Mukil","year":"2010","unstructured":"Mukil Kesavan , Ada Gavrilovska , and Karsten Schwan . 2010 b. On disk scheduling in virtual machines . In Proceedings of the 2nd Workshop on I\/O Virtualization (WIOV\u201910) . 6--6. Mukil Kesavan, Ada Gavrilovska, and Karsten Schwan. 2010b. On disk scheduling in virtual machines. In Proceedings of the 2nd Workshop on I\/O Virtualization (WIOV\u201910). 6--6."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1346256.1346259"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1508244.1508279"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.5555\/1267359.1267361"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1064979.1064984"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the USENIX Annual Technical Conference (ATC\u201908)","author":"Menon Aravind","year":"2008","unstructured":"Aravind Menon and Willy Zwaenepoel . 2008 . Optimizing TCP receive performance . In Proceedings of the USENIX Annual Technical Conference (ATC\u201908) . 85--98. Aravind Menon and Willy Zwaenepoel. 2008. Optimizing TCP receive performance. In Proceedings of the USENIX Annual Technical Conference (ATC\u201908). 85--98."},{"key":"e_1_2_1_32_1","volume-title":"Proceedings of the 2009 Conference on USENIX Annual Technical Conference (ATC\u201909)","author":"Milos Grzegorz","year":"2009","unstructured":"Grzegorz Milos , Derek G. Murray , Steven Hand , and Michael Fetterman . 2009 . Satori: Enlightened page sharing . In Proceedings of the 2009 Conference on USENIX Annual Technical Conference (ATC\u201909) . 1--1. Grzegorz Milos, Derek G. Murray, Steven Hand, and Michael Fetterman. 2009. Satori: Enlightened page sharing. In Proceedings of the 2009 Conference on USENIX Annual Technical Conference (ATC\u201909). 1--1."},{"key":"e_1_2_1_33_1","first-page":"39","article-title":"Addressing TCP\/IP processing challenges using the IA and IXP processors","volume":"7","author":"Minturn Dave","year":"2003","unstructured":"Dave Minturn , Greg Regnier , Jon Krueger , Ravishankar Iyer , and Srihari Makineni . 2003 . Addressing TCP\/IP processing challenges using the IA and IXP processors . Intel Technol. J. 7 , 4, 39 -- 50 . Dave Minturn, Greg Regnier, Jon Krueger, Ravishankar Iyer, and Srihari Makineni. 2003. Addressing TCP\/IP processing challenges using the IA and IXP processors. Intel Technol. J. 7, 4, 39--50.","journal-title":"Intel Technol. J."},{"key":"e_1_2_1_34_1","volume-title":"MPICH: High-performance portable MPI","year":"2013","unstructured":"MPICH2. 2013 . MPICH: High-performance portable MPI . http:\/\/www.mpich.org. MPICH2. 2013. MPICH: High-performance portable MPI. http:\/\/www.mpich.org."},{"key":"e_1_2_1_35_1","unstructured":"NETEM. 2009. Linux network emulation. http:\/\/www.linuxfoundation.org\/collaborate\/workgroups\/networking\/netem.  NETEM. 2009. Linux network emulation. http:\/\/www.linuxfoundation.org\/collaborate\/workgroups\/networking\/netem."},{"key":"e_1_2_1_36_1","unstructured":"Olio. 2011. Apache Olio. http:\/\/incubator.apache.org\/olio\/.  Olio. 2011. Apache Olio. http:\/\/incubator.apache.org\/olio\/."},{"key":"e_1_2_1_37_1","unstructured":"Jon Oltsik and Mark Bowker. 2010. Server virtualization landscape. http:\/\/events.1105govinfo.com\/ events\/vcgsummit2010\/information\/ \/media\/GIG\/GIG%20Events\/2010%20Enterprise%20Architecture\/ Presentations 0\/VCG10 3%201 Oltsik%20Bowker.ashx.  Jon Oltsik and Mark Bowker. 2010. Server virtualization landscape. http:\/\/events.1105govinfo.com\/ events\/vcgsummit2010\/information\/ \/media\/GIG\/GIG%20Events\/2010%20Enterprise%20Architecture\/ Presentations 0\/VCG10 3%201 Oltsik%20Bowker.ashx."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1346256.1346258"},{"key":"e_1_2_1_39_1","doi-asserted-by":"crossref","unstructured":"J. Postel. 1981. Transmission control protocol. RFC 793.  J. Postel. 1981. Transmission control protocol. RFC 793.","DOI":"10.17487\/rfc0793"},{"key":"e_1_2_1_40_1","volume-title":"Dept. of Computer Science","author":"Rangarajan Murali","unstructured":"Murali Rangarajan , Aniruddha Bohra , Kalpana Banerjee , Enrique V. Carrera , Ricardo Bianchini , and Liviu Iftode . 2002. TCP servers: Offloading TCP processing in Internet servers. Tech. rep. DCS-TR-48 , Dept. of Computer Science , Rutgers University , Piscataway, NJ . Murali Rangarajan, Aniruddha Bohra, Kalpana Banerjee, Enrique V. Carrera, Ricardo Bianchini, and Liviu Iftode. 2002. TCP servers: Offloading TCP processing in Internet servers. Tech. rep. DCS-TR-48, Dept. of Computer Science, Rutgers University, Piscataway, NJ."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2004.223"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1592568.1592604"},{"key":"e_1_2_1_43_1","unstructured":"VMware View. 2010. VMware view architecture planning guide. https:\/\/www.vmware.com\/pdf\/view40 architecture planning.pdf.  VMware View. 2010. VMware view architecture planning guide. https:\/\/www.vmware.com\/pdf\/view40 architecture planning.pdf."},{"key":"e_1_2_1_44_1","volume-title":"Proceedings of the 29th Conference on Information Communications (INFOCOM\u201910)","author":"Wang Guohui","unstructured":"Guohui Wang and T. S. Eugene Ng . 2010. The impact of virtualization on network performance of Amazon EC2 data center . In Proceedings of the 29th Conference on Information Communications (INFOCOM\u201910) . 1163--1171. Guohui Wang and T. S. Eugene Ng. 2010. The impact of virtualization on network performance of Amazon EC2 data center. In Proceedings of the 29th Conference on Information Communications (INFOCOM\u201910). 1163--1171."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1383422.1383437"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNET.2006.886335"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.5555\/1973430.1973447"},{"key":"e_1_2_1_48_1","unstructured":"XENCREDIT. 2012. Xen credit scheduler. http:\/\/wiki.xen.org\/wiki\/Credit Scheduler.  XENCREDIT. 2012. Xen credit scheduler. http:\/\/wiki.xen.org\/wiki\/Credit Scheduler."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2287076.2287080"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.5555\/1516124.1516138"}],"container-title":["ACM Transactions on Computer Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2491463","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2491463","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T07:28:49Z","timestamp":1750231729000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2491463"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,8]]},"references-count":50,"aliases":["10.1145\/2518037.2491463"],"journal-issue":{"issue":"3","published-print":{"date-parts":[[2013,8]]}},"alternative-id":["10.1145\/2491463"],"URL":"https:\/\/doi.org\/10.1145\/2491463","relation":{},"ISSN":["0734-2071","1557-7333"],"issn-type":[{"value":"0734-2071","type":"print"},{"value":"1557-7333","type":"electronic"}],"subject":[],"published":{"date-parts":[[2013,8]]},"assertion":[{"value":"2012-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-03-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-08-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}