{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T08:54:37Z","timestamp":1775292877519,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":30,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,7,8]],"date-time":"2022-07-08T00:00:00Z","timestamp":1657238400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,7,8]]},"DOI":"10.1145\/3491418.3535131","type":"proceedings-article","created":{"date-parts":[[2022,7,8]],"date-time":"2022-07-08T16:36:23Z","timestamp":1657298183000},"page":"1-4","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Return on Investment in Research Cyberinfrastructure: State of the Art"],"prefix":"10.1145","author":[{"given":"Craig","family":"Stewart","sequence":"first","affiliation":[{"name":"Department of Computer Science, Luddy School of Informatics, Computing, and Engineering, Indiana University, USA"}]},{"given":"Julie","family":"Wernert","sequence":"additional","affiliation":[{"name":"Indiana University Bloomington, USA"}]},{"given":"Claudia","family":"Costa","sequence":"additional","affiliation":[{"name":"Pervasive Technology Institute, Indiana University, USA"}]},{"given":"David Y.","family":"Hancock","sequence":"additional","affiliation":[{"name":"Indiana University Bloomington, USA"}]},{"given":"Richard","family":"Knepper","sequence":"additional","affiliation":[{"name":"Cornell University, USA"}]},{"given":"Winona","family":"Snapp-Childs","sequence":"additional","affiliation":[{"name":"Indiana University, USA"}]}],"member":"320","published-online":{"date-parts":[[2022,7,8]]},"reference":[{"key":"e_1_3_2_1_1_1","first-page":"87","article-title":"High performance computing instrumentation and research productivity in US universities","volume":"10","author":"Apon Amy","year":"2010","unstructured":"Amy Apon , Stanley Ahalt , Vijay Dantuluri , Constantin Gurdgiev , Moez Limayem , Linh Ngo , and Michael Stealey . 2010 . High performance computing instrumentation and research productivity in US universities . Journal of Information Technology Impact 10 , 2 (2010), 87 \u2013 98 . https:\/\/ssrn.com\/abstract=1679248 Amy Apon, Stanley Ahalt, Vijay Dantuluri, Constantin Gurdgiev, Moez Limayem, Linh Ngo, and Michael Stealey. 2010. High performance computing instrumentation and research productivity in US universities. Journal of Information Technology Impact 10, 2 (2010), 87\u201398. https:\/\/ssrn.com\/abstract=1679248","journal-title":"Journal of Information Technology Impact"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00181-014-0833-7"},{"key":"e_1_3_2_1_3_1","volume-title":"Utilization Patterns, and Approaches to Return on Investment. In Practice and Experience in Advanced Research Computing (PEARC\u201921). ACM","author":"Broude\u00a0Geva Sharon","unstructured":"Sharon Broude\u00a0Geva , Alan Chalker , Curt Hillegas , Donald Petravick , Alan Sill , and Craig Stewart . 2021. Results from a Second Longitudinal Survey of Academic Research Computing and Data Center Usage: Expenditures , Utilization Patterns, and Approaches to Return on Investment. In Practice and Experience in Advanced Research Computing (PEARC\u201921). ACM , New York, NY, USA , Article 41, 4\u00a0pages. https:\/\/doi.org\/10.1145\/3437359.3465589 10.1145\/3437359.3465589 Sharon Broude\u00a0Geva, Alan Chalker, Curt Hillegas, Donald Petravick, Alan Sill, and Craig Stewart. 2021. Results from a Second Longitudinal Survey of Academic Research Computing and Data Center Usage: Expenditures, Utilization Patterns, and Approaches to Return on Investment. In Practice and Experience in Advanced Research Computing (PEARC\u201921). ACM, New York, NY, USA, Article 41, 4\u00a0pages. https:\/\/doi.org\/10.1145\/3437359.3465589"},{"key":"e_1_3_2_1_4_1","volume-title":"HPC. Retrieved","author":"Forum C\u00a0User","year":"2022","unstructured":"HP C\u00a0User Forum . 2022 . Economic Models Linking HPC and ROI . HPC. Retrieved May 11, 2022 from https:\/\/www.hpcuserforum.com\/ROI\/ HPC\u00a0User Forum. 2022. Economic Models Linking HPC and ROI. HPC. Retrieved May 11, 2022 from https:\/\/www.hpcuserforum.com\/ROI\/"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/898\/5\/052014"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-010-9116-9"},{"key":"e_1_3_2_1_7_1","volume-title":"Cost Accounting: Foundations and Evolutions. Cengage Learning","author":"Kinney R.","year":"2011","unstructured":"Michael\u00a0 R. Kinney and Cecily\u00a0 A. Raiborn . 2011 . Cost Accounting: Foundations and Evolutions. Cengage Learning , Boston, MA . Michael\u00a0R. Kinney and Cecily\u00a0A. Raiborn. 2011. Cost Accounting: Foundations and Evolutions. Cengage Learning, Boston, MA."},{"key":"e_1_3_2_1_8_1","unstructured":"Lars Mieritz and Bill Kirwin. 2005. Defining Gartner total cost of ownership. Gartner. https:\/\/www.gartner.com\/en\/documents\/487157  Lars Mieritz and Bill Kirwin. 2005. Defining Gartner total cost of ownership. Gartner. https:\/\/www.gartner.com\/en\/documents\/487157"},{"key":"e_1_3_2_1_9_1","volume-title":"Storage and Analysis. SC14 Conference","author":"Newby Greg","year":"2015","unstructured":"Greg Newby , Amy Apon , Nick Berente , Rudolph Eigenmann , Susan Fratkin , David Lifka , and Craig\u00a0 A Stewart . 2015 . Return on Investment from Academic Supercomputing: SC14 Panel. In Supercomputing 2014: The International Conference for High Performance Computing, Networking , Storage and Analysis. SC14 Conference , New Orleans, LA. https:\/\/hdl.handle.net\/ 2022\/19242 Greg Newby, Amy Apon, Nick Berente, Rudolph Eigenmann, Susan Fratkin, David Lifka, and Craig\u00a0A Stewart. 2015. Return on Investment from Academic Supercomputing: SC14 Panel. In Supercomputing 2014: The International Conference for High Performance Computing, Networking, Storage and Analysis. SC14 Conference, New Orleans, LA. https:\/\/hdl.handle.net\/2022\/19242"},{"key":"e_1_3_2_1_10_1","volume-title":"CNBC. Retrieved","author":"Novet Jordan","year":"2020","unstructured":"Jordan Novet . 2020 . Google\u2019s competitor to Microsoft Office is being rebranded as Workspace and getting new pricing tiers . CNBC. Retrieved May 11, 2022 from https:\/\/www.cnbc.com\/2020\/10\/06\/google-g-suite-becomes-workspace-gets-new-pricing-tiers.html Jordan Novet. 2020. Google\u2019s competitor to Microsoft Office is being rebranded as Workspace and getting new pricing tiers. CNBC. Retrieved May 11, 2022 from https:\/\/www.cnbc.com\/2020\/10\/06\/google-g-suite-becomes-workspace-gets-new-pricing-tiers.html"},{"key":"e_1_3_2_1_11_1","volume-title":"RIMS II: an essential tool for regional developers and planners","author":"Bureau of Economic\u00a0Analysis. 2013.","year":"2022","unstructured":"Bureau of Economic\u00a0Analysis. 2013. RIMS II: an essential tool for regional developers and planners . Bureau of Economic Analysis . Retrieved April 1, 2022 from https:\/\/www.bea.gov\/sites\/default\/files\/methodologies\/RIMSII_User_Guide.pdf Bureau of Economic\u00a0Analysis. 2013. RIMS II: an essential tool for regional developers and planners. Bureau of Economic Analysis. Retrieved April 1, 2022 from https:\/\/www.bea.gov\/sites\/default\/files\/methodologies\/RIMSII_User_Guide.pdf"},{"key":"e_1_3_2_1_12_1","volume-title":"Impacts: Strengthening Research at IU and Throughout the World","author":"\u00a0the Vice President\u00a0for Office","year":"2021","unstructured":"Office of \u00a0the Vice President\u00a0for Information\u00a0Technology. 2021 . Impacts: Strengthening Research at IU and Throughout the World . Indiana University . Retrieved Apr 1, 2022 from https:\/\/issuu.com\/iu-itco\/docs\/rtar21_digital Office of\u00a0the Vice President\u00a0for Information\u00a0Technology. 2021. Impacts: Strengthening Research at IU and Throughout the World. Indiana University. Retrieved Apr 1, 2022 from https:\/\/issuu.com\/iu-itco\/docs\/rtar21_digital"},{"key":"e_1_3_2_1_13_1","volume-title":"Executive Guide: Measuring Performance and Demonstrating Results of Information Technology Investments. Retrieved","author":"United States General\u00a0Accounting Office.","year":"1998","unstructured":"United States General\u00a0Accounting Office. 1998 . Executive Guide: Measuring Performance and Demonstrating Results of Information Technology Investments. Retrieved April 1, 2022 from https:\/\/www.gao.gov\/assets\/aimd-98-89.pdf United States General\u00a0Accounting Office. 1998. Executive Guide: Measuring Performance and Demonstrating Results of Information Technology Investments. Retrieved April 1, 2022 from https:\/\/www.gao.gov\/assets\/aimd-98-89.pdf"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-019-09345-0"},{"key":"e_1_3_2_1_15_1","volume-title":"https:\/\/hdl.handle.net\/2022\/425","author":"Peebles S","year":"2001","unstructured":"Christopher\u00a0 S Peebles , Craig\u00a0 A Stewart , Brian\u00a0 D Voss , and Sue\u00a0 B Workman . 2001. Measuring Quality , Cost, and Value of IT Services (AQC). ( 2001 ). https:\/\/hdl.handle.net\/2022\/425 Christopher\u00a0S Peebles, Craig\u00a0A Stewart, Brian\u00a0D Voss, and Sue\u00a0B Workman. 2001. Measuring Quality, Cost, and Value of IT Services (AQC). (2001). https:\/\/hdl.handle.net\/2022\/425"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/CCGRID.2018.00040"},{"key":"e_1_3_2_1_17_1","volume-title":"Hyperion Research. Retrieved","author":"Research Hyperion","year":"2022","unstructured":"Hyperion Research . 2022. Home Page . Hyperion Research. Retrieved April 1, 2022 from https:\/\/hyperionresearch.com Hyperion Research. 2022. Home Page. Hyperion Research. Retrieved April 1, 2022 from https:\/\/hyperionresearch.com"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1080\/07421222.1992.11517939"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-011-9146-y"},{"key":"e_1_3_2_1_20_1","volume-title":"Practice and Experience in Advanced Research Computing (PEARC\u201921)","author":"Smith Preston","unstructured":"Preston Smith and Stephen Lien\u00a0Harrell . 2021. Research Computing on Campus-Application of a Production Function to the Value of Academic High-Performance Computing . In Practice and Experience in Advanced Research Computing (PEARC\u201921) . ACM , Boston, MA , 1\u20138. https:\/\/doi.org\/10.1145\/3437359.3465564 10.1145\/3437359.3465564 Preston Smith and Stephen Lien\u00a0Harrell. 2021. Research Computing on Campus-Application of a Production Function to the Value of Academic High-Performance Computing. In Practice and Experience in Advanced Research Computing (PEARC\u201921). ACM, Boston, MA, 1\u20138. https:\/\/doi.org\/10.1145\/3437359.3465564"},{"key":"e_1_3_2_1_21_1","volume-title":"Humans in the Loop: Enabling and Facilitating Research on Cloud Computing (Chicago, IL, USA) (HARC \u201919)","author":"Stewart A","unstructured":"Craig\u00a0 A Stewart , Amy\u00a0 W Apon , David\u00a0 Y Hancock , Thomas Furlani , Alan Sill , Julie Wernert , David Lifka , Nicholas Berente , Thomas Cheatham , and Shawn\u00a0 D Slavin . 2019. Assessment of Non-Financial Returns on Cyberinfrastructure: A Survey of Current Methods . In Humans in the Loop: Enabling and Facilitating Research on Cloud Computing (Chicago, IL, USA) (HARC \u201919) . ACM , New York, NY, USA , 10\u00a0pages. https:\/\/doi.org\/10.1145\/3355738.3355749 10.1145\/3355738.3355749 Craig\u00a0A Stewart, Amy\u00a0W Apon, David\u00a0Y Hancock, Thomas Furlani, Alan Sill, Julie Wernert, David Lifka, Nicholas Berente, Thomas Cheatham, and Shawn\u00a0D Slavin. 2019. Assessment of Non-Financial Returns on Cyberinfrastructure: A Survey of Current Methods. In Humans in the Loop: Enabling and Facilitating Research on Cloud Computing (Chicago, IL, USA) (HARC \u201919). ACM, New York, NY, USA, 10\u00a0pages. https:\/\/doi.org\/10.1145\/3355738.3355749"},{"key":"e_1_3_2_1_22_1","volume-title":"Metrics of Financial Effectiveness: Return on Investment in XSEDE","author":"Stewart A","unstructured":"Craig\u00a0 A Stewart , Claudia\u00a0 M Costa , Julie\u00a0 A Wernert , David\u00a0 Y Hancock , Donald\u00a0 F McMullen , Philip Blood , Robert Sinkovits , Susan Mehringer , Richard Knepper , Jeremy Fischer , Marques Bland , Gary Rogers , Peter Couvares , Terry Campbell , Harmony Jankowski , Winona Snapp-Childs , and John Towns . 2022. Metrics of Financial Effectiveness: Return on Investment in XSEDE , A National Cyberinfrastructure Coordination and Support Organization . In Practice and Experience in Advanced Research Computing (PEARC\u201922). ACM, Boston, Massachusetts, In press. https:\/\/doi.org\/10.1145\/3491418.3530287 10.1145\/3491418.3530287 Craig\u00a0A Stewart, Claudia\u00a0M Costa, Julie\u00a0A Wernert, David\u00a0Y Hancock, Donald\u00a0F McMullen, Philip Blood, Robert Sinkovits, Susan Mehringer, Richard Knepper, Jeremy Fischer, Marques Bland, Gary Rogers, Peter Couvares, Terry Campbell, Harmony Jankowski, Winona Snapp-Childs, and John Towns. 2022. Metrics of Financial Effectiveness: Return on Investment in XSEDE, A National Cyberinfrastructure Coordination and Support Organization. In Practice and Experience in Advanced Research Computing (PEARC\u201922). ACM, Boston, Massachusetts, In press. https:\/\/doi.org\/10.1145\/3491418.3530287"},{"key":"e_1_3_2_1_23_1","first-page":"1","article-title":"Assessment of financial returns on investments in cyberinfrastructure facilities: A survey of current methods. In Practice and Experience in Advanced Research Computing (PEARC\u201919). ACM","author":"Stewart A","year":"2019","unstructured":"Craig\u00a0 A Stewart , David\u00a0 Y Hancock , Julie Wernert , Thomas Furlani , David Lifka , Alan Sill , Nicholas Berente , Donald\u00a0 F McMullen , Thomas Cheatham , Amy Apon , Ron Payne , and Shawn\u00a0 D Slavin . 2019 . Assessment of financial returns on investments in cyberinfrastructure facilities: A survey of current methods. In Practice and Experience in Advanced Research Computing (PEARC\u201919). ACM , Chicago , IL , 1 \u2013 8 . https:\/\/doi.org\/10.1145\/3332186.3332228 10.1145\/3332186.3332228 Craig\u00a0A Stewart, David\u00a0Y Hancock, Julie Wernert, Thomas Furlani, David Lifka, Alan Sill, Nicholas Berente, Donald\u00a0F McMullen, Thomas Cheatham, Amy Apon, Ron Payne, and Shawn\u00a0D Slavin. 2019. Assessment of financial returns on investments in cyberinfrastructure facilities: A survey of current methods. In Practice and Experience in Advanced Research Computing (PEARC\u201919). ACM, Chicago, IL, 1\u20138. https:\/\/doi.org\/10.1145\/3332186.3332228","journal-title":"Chicago"},{"key":"#cr-split#-e_1_3_2_1_24_1.1","doi-asserted-by":"crossref","unstructured":"Craig\u00a0A Stewart David\u00a0Y Hancock Julie\u00a0A Wernert Matt\u00a0R Link Nancy Wilkins-Diehr Therese Miller Kelly Gaither and Winona Snapp-Childs. 2018. Return on Investment for Three Cyberinfrastructure Facilities: A Local Campus Supercomputer the NSF-Funded Jetstream Cloud System and XSEDE (the eXtreme Science and Engineering Discovery Environment). In 2018 IEEE\/ACM 11th International Conference on Utility and Cloud Computing (UCC). IEEE Computer Society Los Alamitos CA USA 223-236. https:\/\/doi.org\/10.1109\/UCC.2018.00031 10.1109\/UCC.2018.00031","DOI":"10.1109\/UCC.2018.00031"},{"key":"#cr-split#-e_1_3_2_1_24_1.2","doi-asserted-by":"crossref","unstructured":"Craig\u00a0A Stewart David\u00a0Y Hancock Julie\u00a0A Wernert Matt\u00a0R Link Nancy Wilkins-Diehr Therese Miller Kelly Gaither and Winona Snapp-Childs. 2018. Return on Investment for Three Cyberinfrastructure Facilities: A Local Campus Supercomputer the NSF-Funded Jetstream Cloud System and XSEDE (the eXtreme Science and Engineering Discovery Environment). In 2018 IEEE\/ACM 11th International Conference on Utility and Cloud Computing (UCC). IEEE Computer Society Los Alamitos CA USA 223-236. https:\/\/doi.org\/10.1109\/UCC.2018.00031","DOI":"10.1109\/UCC.2018.00031"},{"key":"e_1_3_2_1_25_1","volume-title":"Advanced Methodologies and Technologies in Network Architecture, Mobile Computing, and Data Analytics. IGI Global","author":"Stewart A","year":"2022","unstructured":"Craig\u00a0 A Stewart , Richard Knepper , Matthew\u00a0 R Link , Marlon Pierce , Eric Wernert , and Nancy Wilkins-Diehr . 2019. Cyberinfrastructure , cloud computing, science gateways, visualization, and cyberinfrastructure ease of use . In Advanced Methodologies and Technologies in Network Architecture, Mobile Computing, and Data Analytics. IGI Global , Hershey, Pennsylvania , 157\u2013170. https:\/\/hdl.handle.net\/ 2022 \/21589 Craig\u00a0A Stewart, Richard Knepper, Matthew\u00a0R Link, Marlon Pierce, Eric Wernert, and Nancy Wilkins-Diehr. 2019. Cyberinfrastructure, cloud computing, science gateways, visualization, and cyberinfrastructure ease of use. In Advanced Methodologies and Technologies in Network Architecture, Mobile Computing, and Data Analytics. IGI Global, Hershey, Pennsylvania, 157\u2013170. https:\/\/hdl.handle.net\/2022\/21589"},{"key":"e_1_3_2_1_26_1","volume-title":"Pervasive Technology Labs, and the Research Technologies Division of University Information Technology Services","author":"Stewart A","year":"1999","unstructured":"Craig\u00a0 A Stewart and Therese Miller . 2011. Economic development by the Indiana University Pervasive Technology Institute , Pervasive Technology Labs, and the Research Technologies Division of University Information Technology Services September 1999 \u2013June 2011: a public report. Technical Report. Indiana University . http:\/\/hdl.handle.net\/2022\/13559 Craig\u00a0A Stewart and Therese Miller. 2011. Economic development by the Indiana University Pervasive Technology Institute, Pervasive Technology Labs, and the Research Technologies Division of University Information Technology Services September 1999\u2013June 2011: a public report. Technical Report. Indiana University. http:\/\/hdl.handle.net\/2022\/13559"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2974927.2974956"},{"key":"e_1_3_2_1_28_1","volume-title":"Retrieved","year":"2022","unstructured":"Top500. 2022 . Top500 The List. Top500 . Retrieved Mar 31, 2022 from https:\/\/www.top500.org\/lists\/top500\/ Top500. 2022. Top500 The List. Top500. Retrieved Mar 31, 2022 from https:\/\/www.top500.org\/lists\/top500\/"},{"key":"e_1_3_2_1_29_1","volume-title":"Practice and Experience in Advanced Research Computing (PEARC\u201921) (Boston, MA, USA) (PEARC\u201921). ACM","author":"von Laszewski Gregor","unstructured":"Gregor von Laszewski , Fugang Wang , and Geoffrey\u00a0 C Fox . 2021. Comprehensive Evaluation of XSEDE\u2019s Scientific Impact Using Semantic Scholar Data . In Practice and Experience in Advanced Research Computing (PEARC\u201921) (Boston, MA, USA) (PEARC\u201921). ACM , New York, NY, USA , Article 50, 4\u00a0pages. https:\/\/doi.org\/10.1145\/3437359.3465601 10.1145\/3437359.3465601 Gregor von Laszewski, Fugang Wang, and Geoffrey\u00a0C Fox. 2021. Comprehensive Evaluation of XSEDE\u2019s Scientific Impact Using Semantic Scholar Data. In Practice and Experience in Advanced Research Computing (PEARC\u201921) (Boston, MA, USA) (PEARC\u201921). ACM, New York, NY, USA, Article 50, 4\u00a0pages. https:\/\/doi.org\/10.1145\/3437359.3465601"}],"event":{"name":"PEARC '22: Practice and Experience in Advanced Research Computing","location":"Boston MA USA","acronym":"PEARC '22","sponsor":["SIGAPP ACM Special Interest Group on Applied Computing","SIGHPC ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing"]},"container-title":["Practice and Experience in Advanced Research Computing"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3491418.3535131","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3491418.3535131","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:48Z","timestamp":1750188648000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3491418.3535131"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,8]]},"references-count":30,"alternative-id":["10.1145\/3491418.3535131","10.1145\/3491418"],"URL":"https:\/\/doi.org\/10.1145\/3491418.3535131","relation":{},"subject":[],"published":{"date-parts":[[2022,7,8]]},"assertion":[{"value":"2022-07-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}