NSDI '24 - Autothrottle: A Practical Bi-Level Approach to Resource Management for SLO-Targeted...

NSDI '24 - Autothrottle: A Practical Bi-Level Approach to Resource Management for SLO-Targeted Microservices Zibo Wang, University of Science and Technology of China and Microsoft Research; Pinghe Li, ETH Zurich; Chieh-Jan Mike Liang, Microsoft Research; Feng Wu, University of Science and Technology of China; Francis Y. Yan, Microsoft Research Awarded Outstanding Paper! Achieving resource efficiency while preserving end-user experience is non-trivial for cloud application operators. As cloud applications progressively adopt microservices, resource managers are faced with two distinct levels of system behavior: end-to-end application latency and per-service resource usage. Translating between the two levels, however, is challenging because user requests traverse heterogeneous services that collectively (but unevenly) contribute to the end-to-end latency. We present Autothrottle, a bi-level resource management framework for microservices with latency SLOs (service-level objectives). It architecturally decouples application SLO feedback from service resource control, and bridges them through the notion of performance targets. Specifically, an application-wide learning-based controller is employed to periodically set performance targets—expressed as CPU throttle ratios—for per-service heuristic controllers to attain. We evaluate Autothrottle on three microservice applications, with workload traces from production scenarios. Results show superior CPU savings, up to 26.21% over the best-performing baseline and up to 93.84% over all baselines. View the full NSDI '24 program at https://www.usenix.org/conference/nsdi24/technical-sessions

• #science_technology
• #usenix
• #technology
• #conference
• #open_access