KubeCon + CloudNativeCon India 2025

Is basic monitoring and logging using prometheus and ELK stack enough? Our DevOps automation platform suffered a major Kubernetes outage due to weak observability despite having the basic setup in place.

What Went Wrong?
Our "good enough" setup fell apart when it mattered most. Logs were a mess with no correlation, & tracing was half-baked, leaving us clueless during debugging. One customer’s CI/CD pipeline overwhelmed shared resources and brought everyone down. Aggressive autoscaling overloaded the control plane. Worst of all, a single tenant’s failure affected the entire cluster.

How did we fix it?
We swapped basic Prometheus for a distributed setup that could handle the scale. Workloads got isolated per tenant and team, with multi-tenancy baked in and dedicated monitoring to match. Fine-grained autoscaling & experimentation with tracing tools(Parca/Odigos). The result? A cluster that bends but doesn’t break.

If you want to avoid your own outage horror story, this talk is for you.

As organisations push to accelerate software delivery, Kubernetes has emerged as the backbone of modern platform engineering, enabling businesses to transcend traditional infrastructure and build scalable, high-performance platforms that empower developers to effortlessly deploy, manage, and scale applications.

In this session, I’ll dive into how Kubernetes is revolutionising platform engineering — turning complex infrastructure into self-service environments that drive automation, scalability, and developer efficiency. Through real-world case studies and forward-thinking strategies, I’ll demonstrate how Kubernetes helps organisations unlock unprecedented flexibility, accelerate delivery cycles, and build platforms that scale seamlessly in today’s dynamic landscape.

Do you know about Large Language Models or LLMs? It’s been a while since we’ve seen a technology like this grab everyone’s attention, techies and non-techies alike. Ever since OpenAI’s ChatGPT burst onto the scene, powered by an LLM, AI has become a household name. But for newcomers, especially in the cloud native ecosystem, these concepts can feel overwhelming.
In this talk, we’ll start from the very basics, explaining what LLMs are and busting some common myths. We’ll break down the key terms with simple diagrams and analogies that are easy to digest. You’ll get a clear, beginner-friendly look at how LLMs work, why Kubernetes is perfect for running them, and how they fit together in the cloud-native world.
We’ll even walk through running an LLM on Kubernetes using vLLM for some practical insights and touch on how this opens the door to AIOps. If you’re curious about stepping into the AI ecosystem and understanding its cloud foundations, this talk is for you.

This talk explores how KServe and OPEA are redefining the enterprise AI stack through modular and composable architectures. KServe delivers a scalable, hardware-agnostic Kubernetes native platform for production model deployment. OPEA complements this infrastructure with an open, multi-provider platform that provides optimized GenAI solutions through standardized interfaces. By building on KServe's capabilities, OPEA allows teams to focus on developing innovative applications rather than managing complex infrastructure concerns.
Join us to learn how deconstructing AI systems into specialized Lego blocks leads to more extensible and powerful enterprise AI architectures. We will demonstrate how this separation of concerns speeds up innovation, enabling teams to focus on app development for their app development. We will also discuss practical approaches for implementing this "building block" methodology in your organization's AI strategy.

Modern AI workloads rely on large GPU fleets whose efficient utilisation is crucial due to high costs. However, gathering telemetry from these workloads to optimise performance is challenging because it requires manual instrumentation and adds performance overheads. Further, it does not produce telemetry in a standardised format for commonly used visualisation tools like Prometheus.

This talk explores the potential of leveraging eBPF to capture CUDA calls made to GPUs, including kernel launches and memory allocations. Data from these probes can be used to export Prometheus metrics, facilitating detailed analysis of kernel launch patterns and associated memory usage. This approach offers significant benefits as eBPF imposes minimal overhead and requires no intrusive instrumentation. Our implementation is also open-source and available on GitHub.

Building ML systems that operate on real-time data streams is no easy feat, especially when dealing with complex messaging systems, scaling requirements, and the need for seamless inference. At Intuit, we saw firsthand how these challenges slowed down our ML teams and hindered innovation. That’s why we created Numaflow, a Kubernetes-native open-source platform that empowers teams to easily connect to streaming sources, apply transformations, and run inference at scale—without the typical overhead. In this talk, we’ll share how Numaflow enhances the developer experience, reduces boilerplate, and accelerates deployment of ML workflows. Whether you're a data scientist, ML engineer, or platform builder, this session will offer practical insights into running real-time inference on streaming data, the Intuit way.

As AI-powered cloud-native applications evolve, ensuring trust, compliance, and robustness requires dynamic governance mechanisms that operate seamlessly across distributed environments. This session introduces a multi-layered cloud-native framework that enforces AI guardrails at three critical stages: pre-processing (input validation), inference (real-time bias mitigation), and post-inference (output validation).

By leveraging Kubernetes orchestration, Istio service mesh, and knowledge graphs, the framework enables scalable AI governance that integrates multi-agent coordination, real-time intervention, and traceability to ensure AI decisions remain transparent, auditable, and aligned with compliance requirements.

Attendees will gain insights into cloud-native AI governance patterns, practical deployment strategies, and the role of multi-agent oversight in ensuring compliant, production-ready AI workflows within Kubernetes environments.

AI Agents are autonomously taking decisions, interacting with each other, and ensuring that the user specified deliverable is achieved. In lot of cases, AI Agents are dynamically generating the code to achieve the functionality. This dynamically generated code needs to be guardrailed i.e., an untrusted model could generate malicious code that will have equal access as that of the model itself. The aim of the talk is to create awareness of security issues sorrounding this use-case, explain the existing tooling/frameworks (such as executing in remotely hosted MicroVMs, use of WASM from NVIDIA). Explain the operational issues using such sandboxing mechanism and then put forth an approach leveraging LSM-BPF that combines the power of Linux Security Modules (LSM) with that of eBPF to achieve better sandboxing. KubeArmor, a CNCF Project, would be used to explain how this can be achieved.