Version v1

Events


Thursday 18:30


Pre-registration meetup

Kinvolk Office

Pre-registration meetup with talks where attendees can pickup badges and hangout.


Friday 09:30


Opening

Loft

Friday 09:45


LinuxBoot and booting fast

Use Linux as the bootloader and start the system up fast. - Kuppel

In contrast to most firmware, like UEFI based or the BIOS, Linux is free software, has drivers for everything and is well known to the administrator. So why not use it in the firmware too? Some work has to be done though to fit it into the flash R...

Due to increases in flash ROM chip sizes, it’s possible again to put the Linux kernel and an initrd there, and use Linux as a boot loader. In 2017 the LinuxBoot project joined the Linux Foundation, and several big companies backed it. This talk presents the project, the benefits and gives a short demonstration in conjunction with coreboot. To achieve the goal, the Linux kernel should be adapted for the board to make it as small as possible. So, it’s back to building your own Linux kernel. Additionally, the boot time should not suffer, so methods on how to profile and to improve start-up time are presented.

Little Services, Big Risks

Extending capability-based security models to achieve micro-segmentation for grids of services - Loft

As we isolate functionality into services distributed across networks, we increasingly strain the concept of trust boundaries. Hosts are no longer simply trusted or untrusted, and each host comes with a new foothold for attackers. This risk is cal...

The Confused Deputy Problem has a history going back to early Unix. Attacks on passwd are a classic case. The architecture of passwd is simple: allow unprivileged users to invoke a setuid executable that performs a high-privilege operation under defined preconditions. This means passwd has been deputized to alter the passwords for all users. However, if you can confuse passwd into misusing its ambient authority, you can gain privileges. As a rule, a compromised service can do anything the service itself is allowed to do. One of the answers has been Mandatory Access Control (MAC) like selinux. Firewalls are analogous for networked services. These systems operate by confining which principals can access which files or services. This helps somewhat; passwd has no business updating /boot (for example). However, in cases like Equifax, principals were only talking to the expected systems; the attacker used the web server to exfiltrate data from the database backing the web application, even accessing tables the web application would be expected to access. Just as with passwd, Equifax’s problem was trusting an application to both perform its functionality and enforce authorizatio...

Friday 10:15


Fedora CoreOS

Loft

What exactly is Red Hat up with CoreOS .....and what were they thinking when they announced a Fedora CoreOS? In this talk, we'll briefly look at some of the excellent work pioneered by the Container Linux team around the self-driving, container fo...

Friday 10:45


A debugger from scratch

Loft

At some stage in your programming life you may well have used a debugger, but did you wonder how it was able to step into and control your executable? In this talk we'll see how debuggers work by building one from scratch in a few lines of Go.

In this talk Liz will explore how a debugger gains control of a process with the all-powerful ptrace system call. You'll see how we can find the machine code that corresponds to a line of human-readable source code and vice-versa. She will show how breakpoints are set, explain how the stack gets built up, and demonstrate how to generate a stack trace showing the path to that breakpoint. Even if you know nothing about machine code, you'll leave this talk with a better understanding of how a computer runs an executable, and how a debugger is able to start and stop the executable as you wish.

HTTP tunneling in Go using HTTP/2 streams

And making it highly available with Serf and Raft - Kuppel

This talk describes our experience developing Wormhole Connector, a distributed proxy component that connects external enterprise systems to a Kyma Kubernetes cluster. The connection between the Wormhole Connector and the Kubernetes cluster is bas...

This talk describes our experience developing Wormhole Connector, a distributed proxy component that connects external enterprise systems to a Kyma Kubernetes cluster. The connection between the Wormhole Connector and the Kubernetes cluster is based on HTTP/2, taking advantage of the stream concept to multiplex multiple connections (HTTP/1 or HTTP/2) through one active HTTP/2 connection. It will cover how we use the Serf and Raft libraries to make the Wormhole Connector highly available, our experience using the Go standard library to support HTTP/2 connections, and technical details describing how everything works under the hood.

Friday 11:30


nettools

A collection of network configuration libraries - Kuppel

nettools is a yet-to-be-released project providing low-level libraries for network configuration. Within the scope of the project falls protocols like DHCP, NDP, IPv4 Address Conflict Detection, and IPv4LL. The first library scheduled to be releas...

Resource Control @FB

Functional resource control using cgroup2 - Loft

After years of development and experimentation, we finally have comprehensive OS-level work-conserving resource isolation working and are now in the process of deploying for various applications including workload protection and container stacking...

FB has been actively experimenting with cgroup2 resource control for years. In the process, we developed several kernel and userland mechanisms, fixed numerous isolation issues, and discovered a number of surprising interactions. We finally have comprehensive OS-level work-conserving resource isolation working and are now in the process of refining and deploying the developed comprehensive resource isolation mechanism for various applications such as workload protection and container stacking. Let's take a look at the mistakes, the lessons, the result, and discuss how best this can be integrated into the whole operating system.

Friday 12:00


Being compliant with Open Container Initiative Spec

Kuppel

Open Container Initiative (OCI) started in 2015 to make different implementations of container runtimes and images compliant with well-defined specifications. Together with other folks at Kinvolk, I have been involved in various OCI projects since...

Open Container Initiative (OCI) defines container runtime specs (https://github.com/opencontainers/runtime-spec) as well as container image specs (https://github.com/opencontainers/image-spec) and distribution spec (https://github.com/opencontainers/distribution-spec). There is also runtime-tools (https://github.com/opencontainers/runtime-tools) that helps container runtime to verify compliance of the runtime specifications. The standard container runtime is runc (https://github.com/opencontainers/runc) that is included in multiple high-level container managers like Docker or containerd. Most of the practical issues arise when specification is not clearly defined in the first place, or when container runtimes have own reasons for not being compliant with the specs, or when there’s no consensus in the community how it should proceed. On the other hand, container orchestration systems like Kubernetes have defined their own interfaces such as Container Runtime Interface (CRI). The different interfaces (OCI runtime and CRI) exist at different layers in the software stack. I'll show how CRI depends on OCI and some mismatches between them. In this talk I want to introduce su...

Friday 12:15


Monitoring File System Syscalls in a Distributed Architecture

Loft

In a distributed world, monitoring system calls with kauditd can present challenges. In this talk we will address some of those challenges and give a use case of how we build an event pipeline for monitoring file system events.

With the rise of containers and generic container based operating systems we find ourselves with a large quantity of nodes that do general compute tasks. These nodes produce a large volume of audit data that we can leverage for many tasks. In our use case we wanted a way to monitor all file system changes in ways that we could not do with the existing libraries or tools. In this talk we will describe how we chose to use audit log system to monitor file system changes, how we built our system to scale and the pros and cons we have found from our solution. We will also talk about possible future work with respect to security and execution monitoring.

Friday 12:30


From Physical to Cloud to Container

One system to deploy and manage them all - Kuppel

Uyuni, an opinionated fork of the Spacewalk project, provides open source lifecycle management for today's datacenter. With the help of Salt for configuration management it keeps your workloads up to date and secure.

Uyuni manages all your Linux workloads. It bootstraps physical servers, creates VMs for virtualization and cloud, builds container images, and tracks what runs on your Kubernetes clusters. Uyuni is open source, backed by SUSE Linux, and actively developed. This presentation will give an overview about this project, it's current possibilities for managing datacenters, and briefly talk about future plans.

Friday 12:45


Monitoring Linux Capabilities in the Container using BPF

Loft

Modern container engines such as systemd.nspawn and rkt provide a means of restricting privilege by limiting Linux capabilities. At Facebook, however, the heterogeneity of services and complexity of libraries running inside the container, along wi...

Friday 13:00


Playing with casync @ instagram

Kuppel

In Instagram, we have been experimenting with casync as an alternative package format for deployment of the site. This talks describe our findings

Last year when Lennart presented in ASG about casync, we were excited to check it out. So we checked what was necessary to deploy parts of our sites with it. and we spend some times experimenting... This talk show our results, so far...

Friday 14:15


Fluent Bit: Solving Logging Challenges for Cloud Native Environments

Loft

Logging could be considered simple when dealing with a few applications, but in environments with distributed systems where log data comes from multiple sources and likely in different formats it becomes complex and data analysis harder. In thi...

Path-agnostic binaries, co-installable libraries, and How To Have Nice Things

Kuppel

Portability is a shining goal for all software -- an objective since the beginning of computing through the present. And yet, it also remains illusive. We once sought to develop software "portable" between whole OSes and kernels; today, we've lo...

Computers are <i>general purpose</i> computers -- they can run any program. But theory aside, is that really true? Forget tivoization and the rise of walled-garden app stores -- when considering new software, often one of the first questions we ask ourselves as linux users is "Is it available on ${distro|packman}?"... We've <i>already</i> been doing this to ourselves for decades. Source builds are possible, but remain full of "well actually" moments; containerization hints at the sheer popularity of The Dream of escaping this madness; and yet, most of our day to day business is guided and limited by what's available in a package manager, and our machines as a whole are limited by which package manager we've chosen. Distributions become inherently balkanized: no one can install anyone else's packages with anything short of a full chroot. Why? Nobody *wants* this. The fracturing creeps in from somewhere. Is it something we can fix? What would it look like if we tried to define packages without distros? Packages without the need for management, beyond drag and drop? Packages with *bring your own* management? Easily relocatable, path-agnostic binaries, and co-insta...

Friday 14:45


To Run an App With Guarantees We Must First Create The Universe

Lessons from managing self-contained application runtimes in production - Kuppel

We’ll look at patterns and anti-patterns for self-contained, immutable runtime environment for applications using Habitat, with a focus on special cases, integrated testing and advanced hacks.

Friday 15:15


oomd

A userspace OOM killer - Loft

Running out of memory on a host is a particularly nasty scenario. In the Linux kernel, if memory is being overcommitted, it results in the kernel out-of-memory (OOM) killer kicking in. In this talk, Daniel Xu will cover why the Linux kernel OOM ki...

Out of memory killing has historically happened inside kernel space. On a memory overcommitted linux system, malloc(2) and friends will never fail. However, if an application dereferences the returned pointer and the system has run out of physical memory, the linux kernel is forced to OOM kill one or more processes. This is typically a slow and painful process because the kernel spends an unbounded amount of time swapping in and out pages and evicting the page cache. Furthermore, configuring policy is not very flexible while being somewhat complicated. oomd aims to solve this problem in userspace. oomd leverages cgroupsv2 and newly exposed counters and statistics to monitor a system holistically. oomd takes corrective action in userspace before an OOM occurs in kernel space. Corrective action is configured via a flexible plugin system, in which custom code can be written. By default, this involves killing offending processes. This enables an unparalleled level of flexibility where each workload can have custom protection rules. Furthermore, time spent churning pages in kernelspace is minimized. In practice at Facebook, we've regularly seen 30 minute host lockups go away entir...

Friday 15:45


Container Run-times and Fun-times

Using systemd inside a container @FB - Loft

A dive into the world of running systemd as an *in container* process manager at Facebook.

At @FB we heavily utilize systemd on our servers. But, there's more! We also heavily utilize systemd inside our containers as well! Combined with our btrfs based image deployment mechanism we leverage all the good parts of systemd within our containers. I'll show how we utilize all the various components of systemd within a container, including how we use Portable Services! I'll talk about the philosophy of this design, our approach to building container images with systemd in mind, and our our approach to Runtime Composition of services. Come listen and enjoy a deep dive into how we use btrfs, systemd, and portable services, and what benefit it provides for us across our large container infrastructure.

dbus-broker

Status update - Kuppel

dbus-broker is an implementation of the DBus specification, intended to be a drop-in replacement for the reference implementation on Linux. It is now scheduled to be the default system and user bus in the next Fedora release. This talks shows some...

Friday 16:30


Titus: Adventures in Multi-tenant Scheduling

Kuppel

Titus is a multitenant scheduler that runs a variety of workloads that vary from online workloads which serve customer traffic to big data workloads which perform machine learning. Getting all of these workloads to cooperate on a shared pool of re...

Titus is a system which allows users to submit arbitrary container workloads to the cloud, and get their workloads running across many thousands of cores or more. This comes with a variety of challenges. We attacked this problem with a three-pronged approach. Our approach to scheduling is multi-tenant first. Our scheduler understands different workloads and the fact that different workloads have different Service Level Objectives. In addition to this, it understands the cloud, and the fact it’s a shared control plane. Lastly, we’ve had to teach our scheduler to handling situations during failover, and when scaling up is key versus traditional scheduling. Our approach to systems is evolving. Historically, our fleet was many single-tenant VMs. We’ve attacked systems level multi-tenancy from the multiple perspectives. The first of these involved giving our user the APIs that were as close to what they had on the VM. Subsequently, we’ve tried to enable security mechanisms like seccomp and apparmor that allow us to run nearly any workload on Titus. Lastly, we’re still figuring out resource isolation. Cgroups have come a long way, but there is a long way to go ahead before we c...

Portable Services are Ready to Use

Loft

Portable Services bring certain aspects of containers to classic system service management. Let's discuss them in technical detail!

Portable Services bring certain aspects of containers to classic system service management. With systemd v239 Portable Services are for the first time complete and ready for users to take advantage of. In this talk we'll have a look on the underlying technical concepts, how things fit together and what the precise limitations and benefits are.

Friday 17:00


Thunderbolt 3 hardware enablement for GNU/Linux

Kuppel

Thunderbolt 3 is a high-speed IO technology that can be used to connect docks, graphic cards or other peripherals requiring high speed. However, the mechanism that allows these high speeds also poses a security risk because malicious devices could...

Thunderbolt 3 is a relatively new technology to connect peripherals to a computer. Because it can access the computer's resources directly, it allows for very high speeds: it is fast enough to drive external graphics cards. However, the mechanism that allows these high speeds also poses a security risk because malicious devices could obtain sensitive information from the computer's memory. Version 3 of the Thunderbolt interface therefore provides security levels in order to mitigate the aforementioned security risk that connected devices pose to the system. As a result, devices need to be authorized manually. The talk aims to provide an overview of the Thunderbolt technology and will try to clarify some of the confusing aspects, e.g. the many modes and features of the USB type C connector that Thunderbolt 3 uses. Finally, the talk will show how some tricky user experience problems were solved,

Friday 17:15


Is my system fast?

An introduction of some tools that help to answer this question. - Loft

Computer systems are complex. Most software applications are distributed and expected to scale. That does not make them any simpler. Further, there is the real world all of that is expected to work in. To analyze performance of such a system isn't...

The talk walks down the path of comparing two Linux servers that look pretty much the same from the specs. Introducing test-tools like stress-ng, sysjitter, fio, packetdrill, and friends it is shown how to get a good overview of the systems CPU, memory, disk, and network performance. Since benchmark results are usually time variant a few basic statistical terms are refreshed and used for visualizing those results employing different chart types. The field of performance analysis is wide. There isn't more time in 45 minutes than just to scratch the surface. At the end of the talk the audience will have a few pointers of how to start evaluating their own Linux systems to provide the right performance for their applications.

Friday 17:30


BPF and the future of the kernel extensibility

Kuppel

To bring cool ideas to life, the Linux kernel and user space need to work together. The core kernel is a stable ABI base, a common denominator for everything that builds on top. In contrast, BPF is a specific know-how, a secret sauce of the cool idea. The Linux kernel needs BPF to stay relevant, and BPF has to become friendlier to programmers. This talk explores the steps taken towards this long-term goal, from recently introduced BPF-to-BPF functions and type information to bounded loops, memory allocation, and beyond.

Friday 19:00


Social event

Mein Haus am See

TBA


Saturday 09:30


CRI-O: All the Runtime Kubernetes need

Loft

CRI-O is a brand new container runtime dedicated and optimized to support kubernetes workload. Its goal is to be a stable container runtime tied to kubernetes releases, replacing the docker daemon. Historically every update of Docker has broken Kubernetes. This has led to major rewriting and fixes of Kubernetes, which is understandable since Docker is not primarily for Kubernetes. Kubernetes needs a container runtime dedicated to its specifications. CRI-O, the name comes from the Container Runtime Interface for Open container runtimes, takes advantages of emerging standards like OCI Runtime and Image Specification, as well as open source projects to handle container images (github.com:containers/image, github.com:containers/storage) . This means as these projects advance CRI-O will be able to take advantage of the improvements and features, but all the while guaranteeing that it will not break any functionality required by the Kubernetes CRI. CRI-O works with runc and Clear Containers runtimes. CRI-O was designed from the ground up to satisfy Kubernetes Container Runtime Interface, and currently passes all node and E2E tests. The github repository has been setup to not ...

Scale Your Auditing Events

Kuppel

The Linux Audit daemon is responsible for writing audit records to the disk, which you can then access with ausearch and aureport. However, it turned out that parsing and centralizing these records is not as easy as you would hope. Elastic's new A...

You can also use Auditbeat to detect changes to critical files, like binaries and configuration files, and identify potential security policy violations. This talk shows you what can you do to discover changes, events, and potential security breaches as soon as possible on interactive dashboards. Additionally, we are combining Auditd events with logs, which are security relevant.

Saturday 10:00


Peer to peer OS and flatpak updates

Kuppel

Recently, work that we have been doing on Endless OS to allow peer to peer OS and flatpak updates has been reaching maturity and nearing wider deployment. This talk will give an overview of how we support LAN and USB updates of OSTrees, how it fit...

State of systemd @ Facebook

Loft

We'll be covering happenings, learnings and new challenges running and supporting systemd in production on the Facebook fleet throughout the past year.

This talk is a followup to <a href="https://www.youtube.com/watch?v=102kzouLJ2c">systemd @ Facebook — a year later</a> that was presented last year. We'll cover the latest developments, how we're leveraging new systemd features, how we continue rolling systemd on the fleet, and finally discuss a number of interesting case studies.

Saturday 10:30


Container Runtimes: draw some lines

Loft

Future of connecting to the container runtime as docker phases out

Since docker came on the scene in 2013, for many it was the first they'd heard of containers. They existed before, and an have exploded since then. Today if you intend on running containers in production it means at the kubernetes level, though what is happening below that? what does this mean for direct access and on-going developer experience?

Running Android on the Mainline Graphics Stack

Kuppel

It is now possible to run Android ontop of an entirely Open Source Linux Graphics stack, this talk will dig into how you can do it too!

Finally, it is possible to run Android on top of mainline Graphics! The recent addition of DRM Atomic Modesetting and Explicit Synchronization to the kernel paved the way, albeit some changes to the Android userspace were necessary. The Android graphics stack is built on a abstraction layer, thus drm_hwcomposer - a component to connect this abstraction layer to the mainline DRM API - was created. Moreover, changes to MESA and the abstraction layer itself were also needed for a full conversion to mainline. This talk will cover recent developments in the area which enabled Qualcomm, i.MX and Intel based platforms to run Android using the mainline graphics stack.

Saturday 11:00


Passive filesystem verification

(and massage) - Loft

A more generic approach to ensure you have what you'd expect to have

A side effect of the many new ways to package filesystems (here's looking at you, containers!), is that filesystems are being copied around without many of the features that traditional packaging provided (i.e. `rpm -qV ...`). Much progress has been made for reproducible digests of containers. In this talk Vincent Batts will review options for distributing filesystems with reproducibility, and verifying the at-rest outcomes.

Chef in Strange Places

Kuppel

How Facebook uses Chef to manage many different largescale heterogenous environments.

Facebook uses chef to deploy and manage a wide array of environments across our fleet, ranging from network switches to mobile phones to LXC containers to off-network devices. This talk will explore our toolchain for building, testing, and running Chef code in this diverse ecosystem of environments and includes stories about it's evolution and development.

Saturday 11:30


The State of Your Supply Chain

Kuppel

Container security often focuses on runtime best-practices whilst neglecting delivery of the software in the supply chain. Application, library, and OS vulnerabilities are a likely route to data exfiltration, and emerging technologies in the conta...

Treating containers as immutable artefacts and injecting configuration allows us to "upgrade" images by rebuilding and shipping whole software bundles, avoiding configuration drift and state inconsistencies. This makes it possible to constantly patch software, and to easily enforce governance of artefacts both pre- and post-deployment. In this talk we detail an ideal, security-hardened container supply chain, describe the current state of the ecosystem, and dig into specific tools. Grafeas, Kritis, in-toto, Clair, Micro Scanner, TUF, and Notary are covered, and we demo how to gate container image pipelines and deployments on cryptographically verified supply chain metadata.

Flatpak, a technical walkthrough

Loft

Flatpak is a desktop-focused application distribution and deployment system for linux. This talk will walk through the technical details of the core functionallity and explain how it work and why it works that way.

<p> Flatpak has now had a major stable release and the rate of change is slowing down. This is a good time to take a look at how the things work behind the scenes. </p> <p> This talk will go through the technical details behind flatpak, explaining things like ostree, bubblewrap and how flatpak uses this to implement the sandbox and basic flatpak commands. It will also explain in high-level terms how the portals and other desktop integration features work. </p>

Saturday 12:00


The Future of Networking APIs

Kuppel

This talk presents TAPS (Transport Services), a proposed abstraction for a new Networking API, and calls for the Linux community to get involved.

Even decades after its introduction, the Socket API is still the de-facto standard Networking API on many systems. Its simplifying abstraction of a network connection as a file helped making networking easier to integrate into applications in the early days, but by now, networking has become way more complex. We have to deal with asynchronous communication, with the choice between IPv4 and IPv6 addresses, with multiple local network interfaces being available, and more. With the development of new transport protocols, such as QUIC, an application may even be able to choose a different transport protocol than TCP, which fundamentally offers the same service. It would be beneficial to develop a future networking API with a better abstraction, so applications can make use of the different options without having to implement a complex decision logic themselves. The Transport Services (TAPS) Working Group in the IETF is currently developing such an abstraction, which will be proposed as a new standard networking API. It will be fundamentally event-based and object-oriented, presenting a simple Connection and Message abstraction to applications. An application only has to specify ab...

Saturday 12:15


Kexec/Kdump under the hood

How to debug your crashed system - Loft

Kdump is a vital tool for debugging severe kernel crashes, especially if the failure can't be reproduced easily or an direct access to the system is not possible.

Kdump is a vital tool for debugging severe kernel crashes, especially if the failure can't be reproduced easily or an direct access to the system is not possible. When an sever error happens in the kernel, a new crash kernel get loaded which saves the memory of the crashed system. These dump can be used to analyze the state of the machine and hopefully give insights on what has happened. This talks will dive into the internals of kexec and kdump. How the crash kernel get set-up, how it's execution get triggered. We will also look into kexec-tool, the user-space part needed to set up a system to use kdump. Where necessary, the architectural specific details will be explained by looking at the arm64 implementation. This talk is thought for people who want to have an insight into how kdump is working.

Saturday 12:30


2018 Desktop Linux Platform Issues

Will the year of the Linux desktop ever arrive? What can we do to make it happen? - Kuppel

TL;DR: Stop making "Desktop Linux" a moving target by agreeing on a minimal baseline that third-party application can take for granted to exist on each Desktop Linux system.

To make "Desktop Linux" a viable platform (as in: Windows, macOS) to develop against, we need to insist on backward compatibility and either need to find the "least common denominator" by experimentation (as we have done for over a decade now), or get the main Linux distributions (Ubuntu, Fedora, openSUSE, Debian, etc.) to agree on a certain set of infrastructure that can be expected to be "there", in a consistant way, without unnecessary differences. A guaranteed minimal set of infrastructure (available in the default installation of all distributions) with guaranteed backward compatibility is required for "Desktop Linux" to become viable as a platform.

Saturday 14:15


Using Machine Learning to find Linux bugs

Loft

I’d like to show you how to find bugs in Linux systems using machine learning, when paired with the totally seemingly useless and annoying false positives that come out of your integration tests.

I’d like to show you how to find bugs in Linux systems using machine learning, when paired with the totally seemingly useless and annoying false positives that come out of your integration tests. We’ve all been frustrated unreproducible bugs in Linux … And also by stupid test-flakes that show up as failures in integration tests even though nothing related has changed. Both unproducible bugs and test flakes are the result of permutations in timing, load, ordering. I want to prove that unreproducible bugs are the same thing as test flakes. And we have a massive source of data here, millions of records of test flakes. Lets take a look at how the Cockpit project trains bots to correlate these flakes into unsupervised clusters. and automatically make use of the data, identifying real bugs, or simply retriggering tests. We’ll dive into details about Normalized Compression Distance, Unsupervised Clustering, TF-IDF and many other simple techniques used to zero in on the bugs.

Configuration Driven Event Tracing with Traceleft and eBPF

Use cases and implementation challenges - Kuppel

Traceleft is a framework built upon eBPF which allows generation of system events such as file operations and network calls via a configuration driven system. It can act as a foundation for building auditing and incident analysis or monitoring too...

The tiny eBPF VM in the Linux kernel has given us unprecedented flexibility in developing tooling for analyzing systems. In this talk we discuss Traceleft - a new auditing and incident analysis framework that can be used to build monitoring tools for filesystem and network events. Traceleft is a configuration driven framework that latches on to syscalls and other relevant Linux kernel functions identified in Traceleft. The eBPF programs are generated, compiled on the fly and inserted in the kernel, based on a set of JSON based configurations. Traceleft supports tracing network calls, file operations such as opening/closing of a target files, for a given set of processes and the aggregation of these events. Traceleft is built using Go and uses Gobpf library and allows the configurations to be sent over the wire using Protobuf format. Traceleft started as a joint effort between ShiftLeft and Kinvolk to develop an easy, repeatable way to provide a configurable way to audit syscalls and classify security events filtered on processes and files. The tools built over Traceleft can be used for live monitoring and alert generation. Implementing Traceleft presented many challenges. ...

Saturday 15:00


Replacing Docker with Podman

Simple tool for running pods and containers - Loft

This talk will describe all of the reasons for podman, all of its features demonstrate its functionality,

I will cover the background of podman, how we built it, why we built it, I will demonstrate using it in multiple different ways, Running containers building container images Communicating with it via var link, cockpit integration. Communicating with it from a remote machine.

libcapsule

segregated loading of dynamic libraries - Kuppel

libcapsule is a project that allows segregated dynamic linking: Access to the symbols of a library without being exposed to any of the dependencies of that library without requiring recompilation of the binary that pulls it in.

libcapsule's goal is to improve portability of programs that are distributed with runtimes (cf flatpak) that still need access to [some] libraries from the host (eg libGL) while insulating said program from any system libraries outside the runtime other than those it directly requires.

Saturday 15:45


Is Cockpit Secure?

Kuppel

Cockpit makes Linux discoverable. But it's really a Linux session in a web browser, accessing the native system APIs and tools directly from javascript. Does that sound scary? How can we be sure that accessing Linux from a web browser is sec...

Fearless Multimedia Programming

Loft

Whether you are interested in multimedia programming in specific or curious about how Rust programming language can enable you to write almost-bug-free code without having to compromise on efficiency, this talk is for you.

GStreamer is a popular framework of choice for multimedia programming in the Linux world, especially for embedded. Since efficiency is a typical core requirement for embedded solutions, traditionally C/C++ have been the languages of choice for writing GStreamer applications and plugins. Unfortunately, this efficiency comes at the price of safety. Even the most skilled C/C++ developers make mistakes with memory management and the results could potentially be catastrophic. Thread safety is another aspect that is central to multimedia solutions but is extremely difficult to achieve with C/C++. Rust language is designed to be both efficient and safe at the same time. In this talk, Zeeshan will present how GStreamer's Rust bindings not only make multimedia programming a lot safer, easier and fun but also allow developers to write even more efficient code than C/C++ in certain cases. While the talk will be focused on multimedia and GStreamer in specific, most of the concepts presented shall be very generic.

Saturday 16:30


Netboot21: Bootloaders in the 21st Century

User-space bootloaders with LinuxBoot - Loft

Sick of insecure PXE booting over TFTP? Come learn about our efforts to write modern boot loaders in Linux's user space.

LinuxBoot is a project to put Linux kernels into firmware. LinuxBoot means we have the full toolset of modern languages like Go at our fingertips: let's use them! We can now easily develop boot loaders with trivial support for HTTPS-, gRPC- or TPM-based network booting.

Past, present and future of system containers

Kuppel

System containers, the oldest type of containers, focus on running an entire Linux distribution, including all its services in very much the same way it would on a physical system or virtual machine.

System containers come with some unique challenges, users of those containers expect to be able to do pretty much everything that they can on a normal system. This means it’s not possible to restrict those containers quite as much as application containers can be. It also means that there are extra expectations to be met: <ul> <li>Being able to add/remove devices to/from a running container</li> <li>Loading security profiles inside a container</li> <li>Using file capabilities in the container</li> <li>Mounting file systems</li> <li>Proper reporting of uptime, resource consumption and limits</li> <li>Live-migration </ul> In this presentation, we’ll explore some of the existing technologies in use by LXC and LXD to address some of those expectations as well as upcoming kernel and userspace features that will allow system containers to do even more than they do today.

Saturday 17:00


Efficient Network Analytics with BPF/eBPF using Skydive

Loft

Efficient monitoring of large-scale networks poses a delicate balance between capture granularity on the one hand and the imposed overheads and performance penalties on the other. Skydive is an open source real-time network topology and protocol a...

Using systemd to high level languages

Kuppel

There is so much more than you can so, than just starting and stooping your service, when you start to interact with systemd from within your application. Lets find out!!!

We'll explore how can you interact with systemd from your aplication, by describing how pystemd, a python binding for systemd, was created at facebook, and what type things your programs can take advantage if you just interact with systemd more. From executing section of your code securely to create development environment on the fly, There is so much more than you can so, than just starting and stooping your service, when you start to interact with systemd from within your application.

Saturday 17:30


Lightning Talks

Submit Your Talks at the Venue - Loft

Please submit your talks to mpitt [at] redhat [dot] com or to organizers at the venue

Early boot provisioning systems

Kuppel

Early boot provisioning systems aim to enable automated, declarative, immutable patterns for Linux systems. In this talk, I'll discuss the CoreOS Ignition system and illustrate how it works and addresses real-world use cases on bare-metal, cloud p...

Early boot provisioning systems prepare a Linux host by partitioning disks, creating filesystems, and writing systemd units and configs. They aim to enable automated, declarative, immutable infrastructure patterns and shape the way large-scale Linux systems operate. In this talk, I'll discuss the original CoreOS declarative config and Ignition system, share how it works, and how it arose. I'll walk through real-world use cases on bare-metal, cloud providers, and hypervisors to show early boot provisioning concepts and where they shine. I'll review experiences using Ignition and cloud-init and integrating them into Matchbox and the Typhoon Kubernetes distribution. For those designing or integrating early boot provisioners, I'll discuss advanced features, exciting areas for exploration, and my hopes for these systems over time. Now that Container Linux and Flatcar Linux support Ignition and Red Hat has announced Fedora CoreOS and Red Hat CoreOS will as well, early boot provisioning can evolve to benefit more distributions and users.

Saturday 18:00


Closing

Loft

Sunday 10:00


Hackfests (Among others: systemd Hackfest)

Gallerie

The All Systems Go! Hackfest. Everyone is welcome to join, form groups and collectively, or individually, work on and discuss projects. We'll have white boards for planning this during all the conference days. Currently confirmed hackfests: ...