In-person + Virtual
November 6-9
Learn More and Register to Attend

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for KubeCon + CloudNativeCon North America 2023 to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

Please note: This schedule is automatically displayed in Central Standard Time (UTC -6). To see the schedule in your preferred timezone, please select from the drop-down menu to the right, above "Filter by Date." The schedule is subject to change.
Thursday, November 9 • 11:55am - 12:30pm
Under the Hood: Exploring Istio's Lock Contention and Its Impact on Expedia's Compute Platform - Raghav Grover & Timothy Ehlers, Expedia

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.

This case study explores Expedia's investigation into a disruptive incident, uncovering a lock contention in Istio. Join us as we deep dive into the systematic steps taken to rule out the various possibilities, navigating through the challenges posed by limited observability and ultimately identifying the root cause. Discover the impact on performance and business metrics, and the collaborative efforts with the Istio open source maintainers, who promptly acknowledged and addressed the issue. This talk will discuss how at Expedia, Compute platform team built a custom Istio Pilot with added observability and tweaked controller queues that helped narrow down the root cause, and how profiling Istio Pilot uncovered a lock contention that introduced significant delays in Proxy propagation and resulted in outages. Furthermore, this talk will provide details about the interim workaround and its impact Expedia implemented before the solution was available upstream.

avatar for Raghav Grover

Raghav Grover

Senior Software Engineer, Expedia
Senior Software Engineer at ExpediaGroup | Enthusiast about highly scalable and performant web apps, Distributed systems and Kubernetes.
avatar for Timothy Ehlers

Timothy Ehlers

Sr software development engineer, Expedia
Senior Software Engineer at ExpediaGroup. Opsdev engineer for life. I always try things the ops way first then do some dev.

Thursday November 9, 2023 11:55am - 12:30pm CST
W175 (Ground Level)
  Service Mesh