Machine Learning on Kubernetes!

Are you retooling your platform team to enable running LLM/ML workloads on Kubernetes? The water’s so much warmer when you have all of the API and autoscaling features at your disposal. The ability to spin up entire RAG-based pipelines for each ML engineer is truly incredible. But it brings new challenges as well - job/node affinity, GPU allocation, cluster autoscaling with GPU scarcity, etc.

In this BoF session, let’s discuss the war stories and solutions we’ve devised running ML on K8s, and learn from each other’s experience.

This is an open discussion where folks can share thoughts, ask questions, and learn together.