eBPF-Enhanced Complete Observability Solution for Cloud-native Microservices

Microservices have emerged as a popular pattern for developing large-scale applications in cloud environments for their flexibility, scalability, and agility benefits. Furthermore, orchestration services like Kubernetes have simplified the deployment of cloud-native applications. However, monitoring and debugging these complex networked applications has become increasingly challenging, creating additional overheads. Traditionally, observability or monitoring requires developers to instrument their applications to expose metrics, logs, and traces using language-restricted libraries. This approach does not work well in a multi-tenant cloud environment as it cannot monitor processes or containers that are not instrumented or hidden. A critical challenge is managing complexity by consistently instrumenting multiple microservices across application platforms and programming languages. Hence, there is a need for a low-overhead cloud-native solution that provides complete observability for distributed and containerized environments. eBPF is a Linux VM technology that can instrument the host kernel directly and provides out-of-the-box cloud-native observability with negligible performance overheads. This paper proposes an eBPF-based solution that offers complete observability for cloudnative deployments. Further, we compare the performance and effectiveness of our solution with popular observability agents like Node Exporter and cAdvisor. We show that our proposed solution reduces CPU overheads by up to 210 times while requiring up to 159 % less memory than the alternatives. Lastly, we deploy and test our solution on a Chameleon cloud bare metal testbed.