{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,6]],"date-time":"2026-04-06T14:59:05Z","timestamp":1775487545090,"version":"3.50.1"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T00:00:00Z","timestamp":1702944000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T00:00:00Z","timestamp":1702944000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Innovations Syst Softw Eng"],"published-print":{"date-parts":[[2025,6]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Language models play a vital role in various natural language processing tasks, but their training can be computationally intensive and lead to significant carbon emissions. In this study, we explore the effectiveness of timeshifting strategies to mitigate the environmental impact of long-running large language models (LLMs). We develop a simulation tool that estimates carbon emissions for LLMs, enabling developers to make informed decisions prior to running their workloads. By leveraging historical carbon intensity data from WattTime, we investigate the potential benefits and limitations of timeshifting in different locations, considering diverse energy profiles. Our findings demonstrate that timeshifting can substantially reduce emissions, but it is highly dependent on the region\u2019s carbon intensity and energy mix. We present insights into the trade-offs between emissions reduction and workload runtime, acknowledging the need for further advancements in carbon-aware computing practices. Our research contributes to the growing field of sustainable computing and encourages developers to adopt environmentally conscious strategies in language model training.<\/jats:p>","DOI":"10.1007\/s11334-023-00546-x","type":"journal-article","created":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T15:02:30Z","timestamp":1702998150000},"page":"517-531","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Timeshifting strategies for carbon-efficient long-running large language model training"],"prefix":"10.1007","volume":"21","author":[{"given":"Akshaya","family":"Jagannadharao","sequence":"first","affiliation":[]},{"given":"Nicole","family":"Beckage","sequence":"additional","affiliation":[]},{"given":"Dawn","family":"Nafus","sequence":"additional","affiliation":[]},{"given":"Scott","family":"Chamberlin","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,12,19]]},"reference":[{"key":"546_CR1","doi-asserted-by":"crossref","unstructured":"Strubell E, Ganesh A, McCallum A (2020) Energy and policy considerations for modern deep learning research. In: Proceedings of the AAAI conference on artificial intelligence, VOL 34(09), PP 13,693\u201313,696","DOI":"10.1609\/aaai.v34i09.7123"},{"key":"546_CR2","doi-asserted-by":"crossref","unstructured":"Strubell E, Ganesh A, McCallum A (2019) Energy and policy considerations for deep learning in NLP. arXiv preprint arXiv:1906.02243","DOI":"10.18653\/v1\/P19-1355"},{"key":"546_CR3","unstructured":"Green Software Foundation, Green Software Practitioner: Carbon Awareness. [Online]. Available: https:\/\/learn.greensoftware.foundation\/carbon-awareness"},{"key":"546_CR4","doi-asserted-by":"crossref","unstructured":"Tiple V (2020) Recommendations on the European commission\u2019s white paper on artificial intelligence-a European approach to excellence and trust, com (2020) 65 final (the\u2019AI white paper)","DOI":"10.2139\/ssrn.3706099"},{"key":"546_CR5","unstructured":"Stokel-Walker C (2023) Turns out there\u2019s another problem with AI\u2014its environmental toll. The Guardian. [Online]. Available: https:\/\/www.theguardian.com\/technology\/2023\/aug\/01\/techscape-environment-cost-ai-artificial-intelligence"},{"issue":"7","key":"546_CR6","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1109\/MC.2022.3148714","volume":"55","author":"D Patterson","year":"2022","unstructured":"Patterson D, Gonzalez J, H\u00f6lzle U, Le Q, Liang C, Munguia L-M, Rothchild D, So DR, Texier M, Dean J (2022) The carbon footprint of machine learning training will plateau, then shrink. Computer 55(7):18\u201328","journal-title":"Computer"},{"key":"546_CR7","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1016\/j.ecolecon.2015.08.020","volume":"120","author":"R Galvin","year":"2015","unstructured":"Galvin R (2015) The ICT\/electronics question: structural change and the rebound effect. Ecol Econ 120:23\u201331","journal-title":"Ecol Econ"},{"key":"546_CR8","unstructured":"Sanh V, Debut L, Chaumond J, Wolf T (2019) DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108"},{"issue":"1","key":"546_CR9","first-page":"5232","volume":"23","author":"W Fedus","year":"2022","unstructured":"Fedus W, Zoph B, Shazeer N (2022) Switch transformers: scaling to trillion parameter models with simple and efficient sparsity. J Mach Learn Res 23(1):5232-5270","journal-title":"J Mach Learn Res"},{"key":"546_CR10","unstructured":"Choromanski K, Likhosherstov V, Dohan D, Song X, Gane A, Sarlos T, Hawkins P, Davis J, Mohiuddin A, Kaiser L et\u00a0al (2020) Rethinking attention with performers. arXiv preprint arXiv:2009.14794"},{"key":"546_CR11","doi-asserted-by":"crossref","unstructured":"Antonello R, Beckage N, Turek J, Huth A (2021) Selecting informative contexts improves language model finetuning. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing, pp 1072\u20131085","DOI":"10.18653\/v1\/2021.acl-long.87"},{"key":"546_CR12","unstructured":"Tay Y, Dehghani M, Bahri D, Metzler D (2020) Efficient transformers: a survey. arXiv preprint arXiv:2009.06732"},{"key":"546_CR13","unstructured":"Bommasani R, Hudson DA, Adeli E, Altman R, Arora S, von Arx S, Bernstein MS, Bohg J, Bosselut A, Brunskill E et\u00a0al (2021) On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258"},{"key":"546_CR14","doi-asserted-by":"crossref","unstructured":"Bannour N, Ghannay S, N\u00e9v\u00e9ol A, Ligozat A-L (2021) Evaluating the carbon footprint of NLP methods: a survey and analysis of existing tools. In: Proceedings of the second workshop on simple and efficient natural language processing, pp 11\u201321","DOI":"10.18653\/v1\/2021.sustainlp-1.2"},{"key":"546_CR15","unstructured":"Lacoste A, Luccioni A, Schmidt V, Dandres T (2019) Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700"},{"issue":"12","key":"546_CR16","doi-asserted-by":"publisher","first-page":"2100707","DOI":"10.1002\/advs.202100707","volume":"8","author":"L Lannelongue","year":"2021","unstructured":"Lannelongue L, Grealey J, Inouye M (2021) Green algorithms: quantifying the carbon footprint of computation. Adv Sci 8(12):2100707","journal-title":"Adv Sci"},{"key":"546_CR17","unstructured":"Patterson D, Gonzalez J, Le Q, Liang C, Munguia L-M, Rothchild D, So D, Texier M, Dean J (2021) Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350"},{"key":"546_CR18","doi-asserted-by":"crossref","unstructured":"Cao Q, Balasubramanian A, Balasubramanian N (2020) Towards accurate and reliable energy measurement of NLP models. arXiv preprint arXiv:2010.05248","DOI":"10.18653\/v1\/2020.sustainlp-1.19"},{"issue":"1","key":"546_CR19","first-page":"10 039","volume":"21","author":"P Henderson","year":"2020","unstructured":"Henderson P, Hu J, Romoff J, Brunskill E, Jurafsky D, Pineau J (2020) Towards the systematic reporting of the energy and carbon footprints of machine learning. J Mach Learn Res 21(1):10 039-10 081","journal-title":"J Mach Learn Res"},{"key":"546_CR20","doi-asserted-by":"crossref","unstructured":"Dodge J, Prewitt T, Tachet Des\u00a0Combes R, Odmark E, Schwartz R, Strubell E, Luccioni AS, Smith NA, DeCario N, Buchanan W (2022) Measuring the carbon intensity of ai in cloud instances. In: 2022 ACM conference on fairness, accountability, and transparency, pp 1877\u20131894","DOI":"10.1145\/3531146.3533234"},{"key":"546_CR21","unstructured":"Touvron H, Lavril T, Izacard G, Martinet X, Lachaux M-A, Lacroix T, Rozi\u00e8re B, Goyal N, Hambro E, Azhar F, Rodriguez A, Joulin A, Grave E, Lample G (2023) LLaMA: open and efficient foundation language models. arXiv:2302.13971"},{"key":"546_CR22","unstructured":"Green Software Foundation, Green Software Practioner: Measurement. [Online]. Available: https:\/\/learn.greensoftware.foundation\/measurement"},{"key":"546_CR23","unstructured":"WattTime, The Power to Choose Clean Energy. [Online]. Available: https:\/\/www.watttime.org\/"},{"key":"546_CR24","unstructured":"O.\u00a0US\u00a0EPA, Download data. May 2022. [Online]. Available: https:\/\/www.epa.gov\/egrid\/download-data"},{"key":"546_CR25","unstructured":"California Energy Commission, 2021 Total System Electric Generation. [Online]. Available: https:\/\/www.energy.ca.gov\/data-reports\/energy-almanac\/california-electricity-data\/2021-total-system-electric-generation"},{"key":"546_CR26","unstructured":"\u201cCodeCarbon.\u201d [Online]. Available: https:\/\/mlco2.github.io\/codecarbon\/"},{"key":"546_CR27","unstructured":"EPA Clean Air Markes Program Data (CAMPD), Power plant emissions, compliance, and allowance data. [Online]. Available: https:\/\/campd.epa.gov"},{"key":"546_CR28","unstructured":"WattTime, Marginal Emissions Methodology. [Online]. Available: https:\/\/www.watttime.org\/marginal-emissions-methodology\/"},{"key":"546_CR29","unstructured":"EPA Office of Air and Radiation (OAR), Greenhouse Gas Equivalencies Calculator. 2015. [Online]. Available: https:\/\/www.epa.gov\/energy\/greenhouse-gas-equivalencies-calculator"},{"issue":"2","key":"546_CR30","doi-asserted-by":"publisher","first-page":"1270","DOI":"10.1109\/TPWRS.2022.3173250","volume":"38","author":"A Radovanovi\u0107","year":"2023","unstructured":"Radovanovi\u0107 A, Koningstein R, Schneider I, Chen B, Duarte A, Roy B, Xiao D, Haridasan M, Hung P, Care N, Talukdar S, Mullen E, Smith K, Cottman M, Cirne W (2023) Carbon-aware computing for datacenters. IEEE Trans Power Syst 38(2):1270-1280","journal-title":"IEEE Trans Power Syst"}],"container-title":["Innovations in Systems and Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11334-023-00546-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11334-023-00546-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11334-023-00546-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T03:05:03Z","timestamp":1750302303000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11334-023-00546-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,19]]},"references-count":30,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["546"],"URL":"https:\/\/doi.org\/10.1007\/s11334-023-00546-x","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-3225810\/v1","asserted-by":"object"}]},"ISSN":["1614-5046","1614-5054"],"issn-type":[{"value":"1614-5046","type":"print"},{"value":"1614-5054","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,19]]},"assertion":[{"value":"1 August 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 November 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 December 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors of this paper are full-time employees of Intel Corporation. The research project received no external sources of funding.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}}]}}