KubeCon + CloudNative North America 2023

Orchestrating complex workflows with heterogeneous components presents challenges that are compounded in ephemeral environments. For example, training of large ML models requires efficiently managing a significant number of expensive accelerators, and building on-demand HPC systems can mean composing applications and services. For both, efficient job orchestration is critical to ensure scalability and high resource utilization. This talk introduces the JobSet API (sigs.k8s.io/jobset) that lays the foundation to automate the setup of these designs. We will first demonstrate how JobSet is used to deploy training workloads using common frameworks like Pytorch, and present results from large scale training experiments on thousands of TPU chips. We then show using JobSet to automate the arduous task of setting up HPC systems on-demand, and creating common environments for experimental comparison.