Traceloop for systemd and Kubernetes + Inspektor Gadget
2019-09-20, 10:30–11:10 (Europe/Berlin), Loft

Presenting traceloop, a “time travel” tracing tool to trace system calls in cgroups using BPF and overwritable ring buffers.


Many people use the “strace” tool to synchronously trace system calls using ptrace. Traceloop similarly traces system calls but asynchronously in the background, using BPF and tracing per cgroup. I’ll show how it can be integrated with systemd and with Kubernetes via Inspektor Gadget.

Traceloop's traces are recorded in a fast, in-memory, overwritable ring buffer like a flight recorder. As opposed to “strace”, the tracing could be permanently enabled on systemd services or Kubernetes pods and inspected in case of a crash. This is like a always-on “strace in the past”.

Traceloop uses BPF through the gobpf library. Several new features have been added in gobpf for the needs of traceloop: support for overwritable ring buffers and swapping buffers when the userspace utility dumps the buffer.

https://github.com/kinvolk/traceloop
https://github.com/kinvolk/inspektor-gadget
https://github.com/iovisor/gobpf

Slides: https://docs.google.com/presentation/d/1zIZUrTrD7FkS9pHnWz87ZmoLTrO1g9-J_lDMD7E5kdo/edit

Originally from France, Alban currently lives in Berlin where he is a CTO & co-founder at Kinvolk. He is a contributor to rkt, a container runtime for Linux, Weave Scope, a container visualization & monitoring tool, and is actively working on BPF-related projects. Before falling into containers, Alban worked on various projects core to modern Linux; kernel IPC and storage, dbus performance and security, etc. His current technical interests revolve around networking, security, systemd and containers at the lower-levels of the system.