{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T07:44:35Z","timestamp":1777362275290,"version":"3.51.4"},"reference-count":0,"publisher":"Global Journals","license":[{"start":{"date-parts":[[2026,2,9]],"date-time":"2026-02-09T00:00:00Z","timestamp":1770595200000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["GJCST"],"published-print":{"date-parts":[[2026,2,9]]},"abstract":"<jats:p>Large-scale Large Language Model (LLM) inference systems deployed in distributed cloud environments face significant challenges in maintaining low latency, efficient GPU utilization, and energy-aware scheduling across heterogeneous hardware. Traditional Kubernetes-based orchestration frameworks are not optimized for the dynamic memory and compute characteristics of transformer workloads, often resulting in resource fragmentation and increased scheduling latency. This paper proposes Adaptive Resource Orchestration (ARO), a telemetry-driven framework designed for distributed LLM inference in heterogeneous GPU clusters. ARO introduces a Rack Affinity Group (RAG) hierarchical indexing mechanism that reduces scheduling search complexity to O(log N * M) while enabling topology-aware resource placement. The framework integrates multi-objective optimization to balance inference latency, energy efficiency, and GPU utilization. Experimental evaluation on a 32-node heterogeneous GPU cluster (H100, A100, and T4) demonstrates significant improvements over baseline Kubernetes scheduling approaches, including up to 84% reduction in P99 latency and 112% improvement in inference energy efficiency (68 tokens\/J). These results highlight the importance of hardware-aware orchestration for improving performance and energy efficiency in distributed AI infrastructure.<\/jats:p>","DOI":"10.34257\/gjcstb156360","type":"journal-article","created":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T06:54:57Z","timestamp":1777359297000},"source":"Crossref","is-referenced-by-count":0,"title":["Dynamic Resource Orchestration for Distributed LLM Inference in Heterogeneous Kubernetes Clusters"],"prefix":"10.34257","author":[{"given":"Mehul","family":"Vani","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"20174","published-online":{"date-parts":[[2026,2,9]]},"container-title":["Global Journal of Computer Science and Technology"],"original-title":[],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T06:54:57Z","timestamp":1777359297000},"score":1,"resource":{"primary":{"URL":"https:\/\/globaljournals.org\/scholarly-articles\/dynamic-resource-orchestration-for-distributed-llm-inference-in-heterogeneous-kubernetes-clusters\/"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,9]]},"references-count":0,"URL":"https:\/\/doi.org\/10.34257\/gjcstb156360","relation":{},"ISSN":["0975-4350","0975-4172"],"issn-type":[{"value":"0975-4350","type":"print"},{"value":"0975-4172","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,9]]}}}