{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,23]],"date-time":"2025-08-23T00:07:24Z","timestamp":1755907644729,"version":"3.44.0"},"publisher-location":"New York, NY, USA","reference-count":28,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,7,17]],"date-time":"2024-07-17T00:00:00Z","timestamp":1721174400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,7,17]]},"DOI":"10.1145\/3626203.3670524","type":"proceedings-article","created":{"date-parts":[[2024,7,17]],"date-time":"2024-07-17T20:12:20Z","timestamp":1721247140000},"page":"1-8","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Lessons from Benchmarking the AI Tennessee Initiative Resources"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3796-8125","authenticated-orcid":false,"given":"Matthew","family":"Bachstein","sequence":"first","affiliation":[{"name":"Office of Innovative Technologies, University of Tennessee, Knoxville, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3697-1218","authenticated-orcid":false,"given":"Christopher","family":"Gropp","sequence":"additional","affiliation":[{"name":"EECS, University of Tennessee, Knoxville, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2981-9920","authenticated-orcid":false,"given":"Victor","family":"Hazlewood","sequence":"additional","affiliation":[{"name":"Office of Innovative Technologies, University of Tennessee, Knoxville, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0875-5278","authenticated-orcid":false,"given":"Gregory","family":"Peterson","sequence":"additional","affiliation":[{"name":"EECS, University of Tennessee, Knoxville, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,7,17]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Retrieved","author":"City Technical","year":"2023","unstructured":"Technical City. 2023. Intel 300 vs AMD EPYC 9734. Retrieved Feb. 14, 2024 from https:\/\/technical.city\/en\/cpu\/EPYC-9734-vs-300"},{"key":"e_1_3_2_1_2_1","unstructured":"PyTorch Contributors. 2023. PyTorch Documentation. Retrieved March 8 2024 from https:\/\/pytorch.org\/docs\/stable\/index.html"},{"key":"e_1_3_2_1_3_1","volume-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arxiv:1810.04805","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arxiv:1810.04805"},{"key":"e_1_3_2_1_4_1","unstructured":"J. Dongarra and M. Heroux. 2013. Toward a New Metric for Ranking High Performance Computing Systems. Technical Report. Sandia National Laboratories."},{"key":"e_1_3_2_1_5_1","unstructured":"A.\u00a0Paszke et al. 2019. PyTorch: An Imperative Style High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32. Curran Associates Inc. none 8024\u20138035. http:\/\/papers.neurips.cc\/paper\/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf"},{"volume-title":"Proc. of 13th USENIX Conf. on Operating Systems Design and Implementation(OSDI\u201918)","author":"P.","key":"e_1_3_2_1_6_1","unstructured":"P.\u00a0Moritz et al. 2018. Ray: a distributed framework for emerging AI applications. In Proc. of 13th USENIX Conf. on Operating Systems Design and Implementation(OSDI\u201918). The Usenix Association, Online, 561\u2013577."},{"key":"e_1_3_2_1_7_1","unstructured":"P.\u00a0Mattson et al. 2019. MLPerf Training Benchmark. arxiv:1910.01500\u00a0[cs.LG]"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"e_1_3_2_1_9_1","volume-title":"Artificial Intelligence Research Computing Symposium. Retrieved","author":"Hazlewood V.","year":"2023","unstructured":"V. Hazlewood. 2023. Artificial Intelligence Research Computing Symposium. Retrieved January 29, 2024 from https:\/\/oit.utk.edu\/hpsc\/artificial-intelligence-symposium-may-17-2023\/"},{"key":"e_1_3_2_1_10_1","volume-title":"Retrieved","author":"Hazlewood V.","year":"2024","unstructured":"V. Hazlewood. 2024. UTK OIT HPSC ISAAC campus cluster node and storage investments. Retrieved February 14, 2024 from https:\/\/oit.utk.edu\/hpsc\/overview\/node-and-storage-investments\/"},{"key":"e_1_3_2_1_11_1","unstructured":"HuggingFace. 2024. HuggingFace Documentation. Retrieved March 8 2024 from https:\/\/huggingface.co\/docs"},{"key":"e_1_3_2_1_12_1","unstructured":"Intel. 2023. Intel Xeon Platinum 8462Y+ Processor. PC Magazine. Retrieved Feb. 10 2024 from https:\/\/ark.intel.com\/content\/www\/us\/en\/ark\/products\/232383\/intel-xeon-platinum-8462y-processor-60m-cache-2-80-ghz.html"},{"key":"e_1_3_2_1_13_1","volume-title":"Inc. Retrieved","author":"Kan M.","year":"2022","unstructured":"M. Kan. 2022. Nvidia\u2019s Latest GPU Boasts a 4-Nanometer Process From TSMC. PC Magazine, Ziff Davis, Inc. Retrieved Feb. 10, 2024 from https:\/\/www.pcmag.com\/news\/nvidias-latest-gpu-boasts-a-4-nanometer-process-from-tsmc-pcie-gen-5"},{"key":"e_1_3_2_1_14_1","volume-title":"MPI: A Message-Passing Interface Standard Version 4.1. Message Passing Interface Forum. https:\/\/www.mpi-forum.org\/docs\/mpi-4.1\/mpi41-report.pdf","author":"Interface Forum Message Passing","year":"2023","unstructured":"Message Passing Interface Forum. 2023. MPI: A Message-Passing Interface Standard Version 4.1. Message Passing Interface Forum. https:\/\/www.mpi-forum.org\/docs\/mpi-4.1\/mpi41-report.pdf"},{"key":"e_1_3_2_1_15_1","unstructured":"Zachary Mueller. 2022. From PyTorch DDP to Accelerate to Trainer mastery of distributed training with ease. https:\/\/huggingface.co\/blog\/pytorch-ddp-accelerate-transformers"},{"key":"e_1_3_2_1_16_1","volume-title":"Retrieved","author":"NVIDIA.","year":"2023","unstructured":"NVIDIA. 2023. NVIDIA H100 Multi-Instance GPU. Retrieved February 10, 2024 from https:\/\/docs.nvidia.com\/datacenter\/tesla\/mig-user-guide\/index.html"},{"key":"e_1_3_2_1_17_1","volume-title":"Retrieved","author":"NVIDIA.","year":"2023","unstructured":"NVIDIA. 2023. NVIDIA H100 Tensor Core GPU datasheet. Retrieved February 10, 2024 from https:\/\/resources.nvidia.com\/en-us-tensor-core\/nvidia-tensor-core-gpu-datasheet"},{"key":"e_1_3_2_1_18_1","volume-title":"Retrieved","author":"NVIDIA.","year":"2023","unstructured":"NVIDIA. 2023. NVIDIA HPC-Benchmarks 23.10. Retrieved November 15, 2023 from https:\/\/catalog.ngc.nvidia.com\/orgs\/nvidia\/containers\/hpc-benchmarks\/"},{"key":"e_1_3_2_1_19_1","volume-title":"Retrieved","author":"University of Tennessee\u00a0Knoxville.","year":"2024","unstructured":"University of Tennessee\u00a0Knoxville. 2024. AI Tennessee Initiative. Retrieved January 27, 2024 from https:\/\/research.utk.edu\/oried\/research-innovation-initiatives\/ai-tennessee-initiative\/"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","unstructured":"A. Pal A. Barigidad and A. Mustafi. 2020. IMDb Movie Reviews Dataset. IMDB. https:\/\/doi.org\/10.21227\/zm1y-b270","DOI":"10.21227\/zm1y-b270"},{"key":"e_1_3_2_1_21_1","unstructured":"The\u00a0Ray Team. 2024. Ray Documentation. Retrieved March 8 2024 from https:\/\/docs.ray.io\/en\/latest\/"},{"key":"e_1_3_2_1_22_1","volume-title":"Retrieved","author":"Technologies Dell","year":"2022","unstructured":"Dell Technologies. 2022. PowerEdge HPC\/AI Server Announcement. Retrieved Feb. 10, 2024 from https:\/\/www.dell.com\/en-us\/dt\/corporate\/newsroom\/announcements\/detailpage.press-releases\u00a0usa\u00a02022\u00a011\u00a020221114-dell-technologies-advances-high-performance-computing\u2013and\u2013ai-with-dell-poweredge-servers-and-dell-apex-innovation.htm"},{"key":"e_1_3_2_1_23_1","volume-title":"Retrieved","author":"Technologies Dell","year":"2023","unstructured":"Dell Technologies. 2023. Dell 6.4 TB NVMe SSD. Retrieved February 10, 2024 from https:\/\/dl.dell.com\/Manuals\/all-products\/esuprt_data_center_infra_int\/esuprt_data_center_infra_storage_adapters\/dell-poweredge-exp-fsh-nvme-pcie-ssd_Users-Guide7_en-us.pdf"},{"key":"e_1_3_2_1_24_1","volume-title":"Retrieved","author":"Technologies Dell","year":"2023","unstructured":"Dell Technologies. 2023. Dell DDR5-4800 product. Retrieved February 10, 2024 from https:\/\/www.serversupply.com\/MEMORY\/PC5-38400\/64GB\/DELL\/370-AGZR_368128.htm"},{"key":"e_1_3_2_1_25_1","volume-title":"Retrieved","author":"Technologies Dell","year":"2023","unstructured":"Dell Technologies. 2023. Dell PowerEdge XE8640 Technical Guide. Retrieved February 10, 2024 from https:\/\/www.delltechnologies.com\/asset\/en-sg\/products\/servers\/technical-support\/poweredge-xe8640-technical-guide.pdf"},{"key":"e_1_3_2_1_26_1","unstructured":"GitHub Users. 2023. It is SUPER unclear how to run multi-node distributed training with HuggingFace Accelerate. https:\/\/github.com\/huggingface\/accelerate\/issues\/1242"},{"key":"e_1_3_2_1_27_1","unstructured":"GitHub Users. 2023. launch.slurm. https:\/\/gist.github.com\/pacman100\/1cb1f17b2f1b3139a63b764263e70b25"},{"key":"e_1_3_2_1_28_1","unstructured":"HuggingFace Users. 2022. Distributed Training w\/ Trainer. https:\/\/discuss.huggingface.co\/t\/distributed-training-w-trainer\/8151"}],"event":{"name":"PEARC '24: Practice and Experience in Advanced Research Computing","sponsor":["SIGAPP ACM Special Interest Group on Applied Computing","SIGHPC ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing"],"location":"Providence RI USA","acronym":"PEARC '24"},"container-title":["Practice and Experience in Advanced Research Computing 2024: Human Powered Computing"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3626203.3670524","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3626203.3670524","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T12:57:33Z","timestamp":1755867453000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3626203.3670524"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,17]]},"references-count":28,"alternative-id":["10.1145\/3626203.3670524","10.1145\/3626203"],"URL":"https:\/\/doi.org\/10.1145\/3626203.3670524","relation":{},"subject":[],"published":{"date-parts":[[2024,7,17]]},"assertion":[{"value":"2024-07-17","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}