0.14 all-systems-go-2023 2023-09-13 2023-09-14 2 00:05 https://cfp.all-systems-go.io https://cfp.all-systems-go.io/media/all-systems-go-2023/img/ASG_social-avatar_Nl7AMBw.png Europe/Berlin Main Hall Lightning talk 2023-09-13T09:30:00+02:00 09:30 00:05 A welcome session for All Systems Go! all-systems-go-2023-239-opening-session-of-all-systems-go-2023 en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/X89KG9/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/X89KG9/feedback/ Main Hall 35 min talk + 5 min Q&A 2023-09-13T09:45:00+02:00 09:45 00:40 UKIs are a fundamental building block of modern measured and trusted boot chains. Let's have a look at what happened in the area and discuss recently added new concepts, such as "add-ons", new PE sections, build tools and more. all-systems-go-2023-185-unified-kernel-images-ukis- Lennart Poettering en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/ZEVAWH/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/ZEVAWH/feedback/ Main Hall 35 min talk + 5 min Q&A 2023-09-13T10:30:00+02:00 10:30 00:40 In this presentation, we introduce Inspektor Gadget, a tool designed for the creation, deployment, and execution of eBPF programs (gadgets) across Kubernetes and Linux environments. Inspektor Gadget encapsulates eBPF programs into OCI containers, providing well-understood and easily distributable units. We'll delve into Inspektor Gadget's automatic data enrichment process, transforming complex kernel information into high-level, understandable concepts tied to Kubernetes, container runtimes, systemd, etc. This feature bridges the knowledge gap between raw, low-level data and more interpretable information, improving the understanding of system behavior. We will illustrate how to use a simple configuration file to set up a data collection pipeline with Inspektor Gadget, resulting in a Prometheus endpoint or an exposed API. Throughout the talk, we'll demonstrate Inspektor Gadget's features, support across various environments, discuss its operational mechanics, and share insights into the future direction of the project. By presenting at ASG!, our aim is not just to inform the audience of Inspektor Gadget, but also to encourage feedback and stimulate discussions within the eBPF and Linux community. all-systems-go-2023-230-gaining-linux-insights-with-inspektor-gadget-an-ebpf-tool-and-systems-inspection-framework Chris KuehlAlban Crequy en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/ZSTFTF/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/ZSTFTF/feedback/ Main Hall 35 min talk + 5 min Q&A 2023-09-13T11:15:00+02:00 11:15 00:40 Using an image-based OS brings advantages and challenges. One challenge is the customization of a read-only image with additional host-level software and configuration, and how to manage this customization through the lifetime of a machine. For deeper changes in /usr, users might build their own images instead of following the official image updates. For common scenarios, the vendor may choose to offer multiple image flavors. Simpler user customization can live outside of the read-only /usr, scattered as config files and binaries in /etc and /opt. Configuration management tools struggle with reliable (re)configuration because tracking filesystem state is hard. The systemd project now supports a mechanism for extension images. There are two types; system extensions create an overlay for /usr or /opt and configuration extensions create an overlay for /etc. Through the overlay, users can thus change the read-only /usr without building custom OS images. Vendors can also offer their supported flavors as extensions instead of different OS images, even as composable stack where the user can choose optional parts. Users can manage their configuration by replacing the extension images atomically. Since the images bundle all files, this prevents old files lingering around or a system in a half-finished state. The read-only extension images help with setting up attestation and integrity enforcement for their contents. For distributions providing prebuilt initrds (e.g., the Fedora mkosi-initrd proposal), extensions allow initrd customization provided by the distribution or user. The presentation will give an overview, share use cases and examples, and discuss future improvements for extension images. all-systems-go-2023-195-system-and-configuration-extensions-for-image-based-linux-distros-and-beyond /media/all-systems-go-2023/submissions/XLQNDJ/systemd-logo_QXIg6eL.png Kai LükeMaanya GoenkaLuca Boccassi en A recent addition to the systemd toolbox was systemd-sysext for system extensions through overlay images mounted on /usr. Even newer is systemd-confext for configuration extensions through overlay images mounted on /etc. The main use case for systemd-sysext is the customization or deployment of additional software on an image-based OS where /usr is read-only. The use of single images that contain all files allows to reliably manage the changes compared to unpacking files to the root filesystem. Optional dm-verity protection ensures the integrity of the extensions. A simple version matching scheme allows to either couple the extension to the OS version or not. The first case is useful for officially released OS extensions or dynamic linking, the second for static linking and only few assumptions about the host. For systemd-confext the use case is similar as with systemd-sysext but it focuses on configuration in /etc. Here again, the use of single image files makes configuration changes more reliable. Ideally the use of configuration images should allow to have /etc read-only at runtime, following the idea of immutable infra. However, not all software and workflows are prepared for that, and the goal is to introduce different modes for the overlay to, e.g., support ephemeral or persistent changes. Other plans are to set up the overlay mount from the initrd already to have all configuration in place as early as possible, and to improve the live reload behavior through atomic mount operations and system reload actions. The presentation will show how to use systemd-sysext/confext and share some examples from Flatcar Container Linux and an embedded Linux platform for both coupled and decoupled extensions. false https://cfp.all-systems-go.io/all-systems-go-2023/talk/XLQNDJ/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/XLQNDJ/feedback/ Main Hall 20 min talk + 5 min Q&A 2023-09-13T12:00:00+02:00 12:00 00:25 Stopping the old and starting a new service afresh -- that is what service restart is roughly about. We will look what it comprises in more detail from service manager perspective and also from the service's client end. Thus we will look at how FDSTORE API can be used to smooth service restart. Furthermore, we will review how unit instances may provide further distinction between the stopped and the restarted service. Finally, we go through options that the existing service have to adopt these methods. all-systems-go-2023-220-retake-of-service-restarts Michal Koutný en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/FYHCNJ/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/FYHCNJ/feedback/ Main Hall 20 min talk + 5 min Q&A 2023-09-13T12:30:00+02:00 12:30 00:25 systemd v254 introduced a new reboot type: soft-reboot. It shortcuts the reboot process by not restarting the kernel, and instead shutting down userspace, followed by re-exec'ing systemd from the new rootfs, starting everything up again. Not only this allows to save time by virtue of doing less work, but it also allow select resources (File Descriptor Store) and select services that do not use the rootfs (Portable Services) to survive the reboot and continue uninterrupted. This talk will explore the details of this new feature, how it works, why it's useful, what are the shortcomings and how to make full use of it. all-systems-go-2023-184-soft-reboot-atomically-replace-rootfs-and-reboot-userspace-without-kernel-restart Luca Boccassi en In many environments where image-based Linux is used, service interruption intervals are key metrics that need to be minimized as much as possible. On a traditional package-based distributions, the rootfs can be updated piecemeal and userspace services can be restarted one by one - assuming a perfect running dependency tracking system and perfect reliability (need to restart D-Bus? Good luck!). On an image-based system this is obviously not possible, so a typical approach is relying on 'kexec', which loads a new kernel + initrd + rootfs, saving some time from a full reboot by avoiding giving back control to the firmware. But it turns out, it's not fast enough. systemd v254 introduced a new reboot type: soft-reboot. This follows in the kexec footsteps by shortcutting the reboot process, and brings it ever further: the kernel is not restarted at all, and instead userspace is shut down and then systemd is re-exec'ed from the new rootfs, starting up again. Not only this allows to save time by virtue of doing less work, but it also allow resources (File Descriptor Store) and select services that do not use the rootfs (Portable Services) to survive the reboot and continue uninterrupted. This talk will explore the details of this new feature, how it works, why it's useful, what are the shortcomings and how to make full use of it. false https://cfp.all-systems-go.io/all-systems-go-2023/talk/XVV9QY/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/XVV9QY/feedback/ Main Hall 35 min talk + 5 min Q&A 2023-09-13T14:30:00+02:00 14:30 00:40 Let's get you up to speed on Trusted Platform Modules (TPM 2.0) and Linux. Specifically, the various additions to basic Linux userspace, i.e. systemd in our goal to make measured boot a default on Linux. all-systems-go-2023-186-linux-tpms Lennart Poettering en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/HSEJY9/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/HSEJY9/feedback/ Main Hall 20 min talk + 5 min Q&A 2023-09-13T15:15:00+02:00 15:15 00:25 The TPM event log contains a history of all measurements made with the TPM. Complete with some context information for each measurement it is intended to help with recreating the current PCR contents. What was meant as a debugging tool turns out to be of vital importance when trying to remotely attest real life systems. This is mostly because of the overuse of certain PCR and the general mess that is x86 firmware. Sadly, there are many event logs. UEFI keeps one for its measurements and those done by EFI applications like GRUB and shim. If a system is booted in an MLE using tboot the ACM firmware code also maintains an event log that can be accessed via a pointer in an ACPI table. Now, systemd also has an event log that is mixed into the general journal log. Finally Linux IMA maintains it's own event log -- an append-only, in-kernel data structure. On top of that every bootloader or userspace application that wants to measure something into the TPM will also need to maintain an event log. How about we fix that? The talk will sketch out a solution that maintains a unified, global event log of the whole system on disk and exposes an interface for other applications that wish to measure things into the TPM. We'll also fix a race conditions in IMA as well as correctly handle S3 resume w.r.t measured boot while we're at it. all-systems-go-2023-204-an-unified-tpm-event-log-for-linux Kai Michaelis en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/HGMV9U/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/HGMV9U/feedback/ Main Hall 20 min talk + 5 min Q&A 2023-09-13T16:30:00+02:00 16:30 00:25 A short case study on where we are with sandboxing APT; what gaps there are and what technologies we looked at. all-systems-go-2023-198-wip-sandboxing-apt Julian Andres Klode en Downloading packages, verifying packages, installing packages, protecting user data from snoopy or broken maintainer scripts. A package manager has a lot of places that can need some sort of sandboxing. APT currently employs a minimal sandbox using a separate user for downloading, and optionally seccomp. This talk will explore that, the caveats and some more avenues like landlock, running apt in systemd isolation (useful for our apt-based .service units), file descriptor passing into sandbox. false https://cfp.all-systems-go.io/all-systems-go-2023/talk/8CGF9L/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/8CGF9L/feedback/ Main Hall 20 min talk + 5 min Q&A 2023-09-13T17:00:00+02:00 17:00 00:25 The utmp implementation of glibc uses on quite some 64bit architectures a 32bit time variable, which leads to an overflow on 03:14:07 UTC on 19 January 2038. This talk will explain the current work on replacing utmp with logind. all-systems-go-2023-183-y2038-replace-utmp-with-logind Thorsten Kukuk en The year 2038 problem (also known as Y2038) is a time formatting bug on Unix systems with representing times after 03:14:07 UTC on 19 January 2038. This happens with a 32bit time_t, not with a 64bit time_t. The general statement so far has always been that on 64bit systems with a 64bit time_t you are safe with respect to the Y2038 problem. But this isn't correct: on bi-arch systems like x86-64 (so which can execute 64bit and 32bit binaries) glibc defines __WORDSIZE_TIME64_COMPAT32, which leads to the fact, that struct utmp (used for utmp, wtmp and btmp) and struct lastlog uses int32_t instead of time_t. So we have a Y2038 problem, which is not easy fixable, as this would require ABI and on disk format changes. In this talk I will speak about the background, which tools are affected and a radical solution: drop utmp, wtmp, btmp and lastlog completely and make use of systemd-logind and other tools instead. false https://cfp.all-systems-go.io/all-systems-go-2023/talk/3Z7XEE/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/3Z7XEE/feedback/ Main Hall Lightning talk 2023-09-13T17:30:00+02:00 17:30 00:05 Some quick numbers and maybe curiousities from our work on evaluating which libraries need to be rebuilt for 64-bit time_t on armhf in Ubuntu using abi-compliance-checker. all-systems-go-2023-199-64-bit-timet-on-armhf-running-abi-compliance-checker-on-all-of-ubuntu Julian Andres Klode en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/R3SWBQ/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/R3SWBQ/feedback/ Main Hall Lightning talk 2023-09-13T17:35:00+02:00 17:35 00:05 Image based OS updates are the future. One way to handle updates is via content-addressable synchronisation software, like casync and desync. This talk with give a presentation about the two - their overall design, feature set and strengths and weaknesses. It will also demonstrate a real world use-case of them. all-systems-go-2023-209-casync-is-not-dead-or-how-i-learned-to-love-desync Emil Velikov en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/9MVYFU/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/9MVYFU/feedback/ Main Hall Lightning talk 2023-09-13T17:40:00+02:00 17:40 00:05 A quick overview of the work in progress to plumb PID FDs through Linux userspace, to achieve resilience and security improvements all-systems-go-2023-178-pid-fd-ize-all-the-things- Luca Boccassi en Process ID File Descriptors were introduced in Linux v5.3. They allow tracking a process reliably, without risking races and reuse attacks, as they always refer to one single process regardless of the actual PID, so if the process goes away the file descriptor will become invalid, even if a new process with the same PID reappears at the same time. Tracking processes in userspace is needed for various purposes, for example to authenticate actions via Polkit. This has been historically fragile, and various workarounds such as tracking a PID plus a UID plus a start time were put in place. D-Bus implementations also have methods to query a D-Bus' endpoint's PID, UID and GIDs. Recently work has been done to plumb PID FDs through all these components - systemd is able to receive queries asking for the session information or unit information via a PID FD, D-Bus implementations return the PID FD of a D-Bus endpoint via GetConnectionCredentials()/GetConnectionUnixProcessFD() (and they track processes via FD rather than PID), and Polkit allows writing rules authorizing by the systemd service name, which is possible to do safely thanks to using FDs all the way through. This lightning talk will quickly go through these improvements, showing how PID FDs can be used to improve userspace and provide concrete benefits. false https://cfp.all-systems-go.io/all-systems-go-2023/talk/T3LJAM/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/T3LJAM/feedback/ Main Hall Lightning talk 2023-09-13T17:45:00+02:00 17:45 00:05 The journey of developing a Linux platform to require very little in the way of configuration management, and how to virtually eliminate the need to modify code to change configuration. From configuration via scripts and evolving through a couple of configuration management products, we have used the idea of matching actions to timescales to transform how we do configuration management. We now do very little of it, and we have dramatically reduced its complexity. all-systems-go-2023-194-principle-of-least-configuration James Morris en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/PVJQTH/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/PVJQTH/feedback/ Main Hall Lightning talk 2023-09-13T17:50:00+02:00 17:50 00:05 All Systems Go! lightning talk all-systems-go-2023-246-attaching-cpus-via-usb Daniel Maslowski en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/VAY88J/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/VAY88J/feedback/ Main Hall Lightning talk 2023-09-13T17:55:00+02:00 17:55 00:05 All Systems Go! lightning talk all-systems-go-2023-245-tvix-store flokli en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/AKNDS3/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/AKNDS3/feedback/ Main Hall Lightning talk 2023-09-13T18:00:00+02:00 18:00 00:05 All Systems Go! lightning talk all-systems-go-2023-247-carbon-os-homed Adrian Vovk en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/8P7XKH/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/8P7XKH/feedback/ Main Hall Social event 2023-09-13T19:00:00+02:00 19:00 04:00 The social event will take place, once again, at [Haus am See](https://goo.gl/maps/PShjH32z7BJJJB2L9) from 19:00-23:00. Food will be served and drinks tokens will be handed out at the door. **19:00-21:00** - Food and drinks on the ground floor with access to the club are on the lower level **21:00-23:00** - We move to the club area on the lower floor which we have exclusively for more drinks and mingling. The ground floor will be open for non-All Systems Go! folks. You can stay after after 23:00 but after that point there is no official All Systems Go! function and you're own your own. ;) all-systems-go-2023-243-evening-social-event-mein-haus-am-see en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/WATVQQ/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/WATVQQ/feedback/ Dome 35 min talk + 5 min Q&A 2023-09-13T10:30:00+02:00 10:30 00:40 At Meta, we've been working to add encryption support to btrfs, with exciting implications for per-container security. Traditionally encryption has either dealt with whole disks, with LUKS, or with a few filesystems: ext4, f2fs, ubifs, and ceph, lacking in advanced volume management. Btrfs has several features these filesystems don't: deduplicating/reflinking identical data, subvolume/snapshot management, and integrated checksumming. These features allow giving containers their own encrypted subvolume with a key only loaded when the container is running, preventing container storage from being read while turned off, and making deletion of expired containers' storage secure. all-systems-go-2023-221-encrypted-btrfs-subvolumes-keeping-container-storage-safe Sweet Tea Dorminy en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/ZJDHRA/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/ZJDHRA/feedback/ Dome 35 min talk + 5 min Q&A 2023-09-13T11:15:00+02:00 11:15 00:40 With the introduction of "Forensic Container Checkpointing" in Kubernetes 1.25 it is possible to checkpoint containers. The ability to checkpoint containers opens up many new use cases. Containers can be migrated without loosing the state of the container, fast startup from existing checkpoints, using spot instances more effective. The primary use case, based on the title of the Kubernetes enhancement proposal, is the forensic analysis of the checkpointed containers. In this session I want to introduce the different possible use cases of "Forensic Container Checkpointing" with a focus on how to perform forensic analysis on the checkpointed containers. The presented use cases and especially the forensic analysis will be done as a live demo giving the audience a hands on experience. all-systems-go-2023-177-forensic-container-checkpointing-and-analysis Adrian Reber en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/FZNLRT/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/FZNLRT/feedback/ Dome 20 min talk + 5 min Q&A 2023-09-13T12:00:00+02:00 12:00 00:25 strace is a traditional userspace tracer utility for Linux, implemented using ptrace API. Despite of the abundance of various kernel tracing interfaces nowadays, there are certain classes of tasks that are still better served by strace. In this talk the maintainer of strace will provide examples of such tasks. all-systems-go-2023-228-why-would-you-still-want-to-use-strace-in-2023- /media/all-systems-go-2023/submissions/GUVYJ7/Straus-196_hAmMwdH.png Dmitry LevinEugene Syromiatnikov en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/GUVYJ7/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/GUVYJ7/feedback/ Dome 20 min talk + 5 min Q&A 2023-09-13T12:30:00+02:00 12:30 00:25 Let's discuss about `bpfilter`, a userspace daemon that empowers services to create efficient packet-filtering BPF programs using a high-level representation of filtering rules. all-systems-go-2023-196-bpfilter-a-bpf-based-packet-filtering-framework Quentin Deslandes en For a significant period, `bpfilter` wasn't more than an empty [usermode helper](https://cateee.net/lkddb/web-lkddb/STATIC_USERMODEHELPER.html) and an [abandoned patch series](https://lore.kernel.org/bpf/20210829183608.2297877-1-me@ubique.spb.ru). However, it has recently undergone active development as a userspace daemon, which can be found on GitHub at [https://github.com/facebook/bpfilter](https://github.com/facebook/bpfilter). This daemon now offers userspace services a swift and user-friendly interface to generate packet-filtering BPF programs dynamically. This discussion aims to provide further insights into `bpfilter`, including its current capabilities, performance, and ongoing development efforts. false https://cfp.all-systems-go.io/all-systems-go-2023/talk/BKLNWP/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/BKLNWP/feedback/ Dome 35 min talk + 5 min Q&A 2023-09-13T14:30:00+02:00 14:30 00:40 This talk will discuss new features provided by the new kernel mount API interface all-systems-go-2023-244-new-mount-api Christian Brauner en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/NYLYDK/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/NYLYDK/feedback/ Dome 35 min talk + 5 min Q&A 2023-09-13T15:15:00+02:00 15:15 00:40 Despite being ordinary computers with an ASIC for switching, in reality network hardware must still be treated differently from normal servers. In recent years a lot has improved, and vendors offer white box switches, allowing users to install a (network) operating system of their choice. Of course, the NOS needs to support the firmware interface for the particular ASIC, and this is not standardized: swtitchdev, DSA, SAI – none of them supporting all devices. Due to SONiC dominance, a lot of vendors seem to support SAI (Switch Abstraction Interface). But SAI requires a proprietary external Linux kernel module. On the NOS side, Open Network Linux was abandoned, and Azure’s SONiC is the new popular kid on the block, running a Docker daemon. There are other differences in the network hardware ecosystem: For example ONIE as the bootloader environment. Also working with upstream and using established software developing practices are lacking, resulting in a maintenance burden. Projects like DENT or OpenWrt go one step further by only supporting upstream Linux kernel interfaces, but now dentOS is also going to support SAI. This talk gives a short introduction into the network operating systems, and then focuses on DENT with the ONL fork dentOS, and shares experiences. Curiously, problems how to treat firmware blobs and discussions about what distribution to use as a base, are not unknown to these projects either. all-systems-go-2023-236-disaggregated-networks-is-network-hardware-special- Paul Menzel en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/JVGVHG/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/JVGVHG/feedback/ Dome 20 min talk + 5 min Q&A 2023-09-13T16:30:00+02:00 16:30 00:25 In light of the climate crises, and despite hardware getting faster and faster, fully powering down systems and back on on demand – the obvious choice – is still inconvenient, as boot times are still very long. Even ChromeOS still has not lowered its limit from ten seconds since years. Show the current status of the hobby project on x86 hardware, and give an overview of recent Linux kernel developments getting rid some of the delays. all-systems-go-2023-237-booting-fast-why-does-power-on-to-login-still-last-longer-than-one-second- Paul Menzel en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/HZY3K8/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/HZY3K8/feedback/ Dome 20 min talk + 5 min Q&A 2023-09-13T17:00:00+02:00 17:00 00:25 A walkthrough of an interesting use case for the `FICLONE` ioctl: cloning file data into a tar archive, and cloning files out of it again. "Free" archiving and unarchiving at zero-copy speeds! Topics: - Copy-on-write and the `FICLONE` ioctl - The ancient `tar` format - A trick for adding arbitrary padding to the `tar` format in order to force file system page alignment - How to avoid symlink attacks and other TOCTOU issues, using the fairly recently introduced (linux 5.6) `openat2` system call. - An interesting bug in GNU tar At the end you'll receive a free autographed copy of [deduptar](https://git.sr.ht/~nullenenenen/deduptar/tree/master/item/README.md) to use for party tricks. 🥳 all-systems-go-2023-225-making-a-magic-deduplicating-tar-using-the-ficlone-ioctl /media/all-systems-go-2023/submissions/99PZDY/tar_pzOsyxR.jpg Wicher Minnaard en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/99PZDY/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/99PZDY/feedback/ Main Hall 35 min talk + 5 min Q&A 2023-09-14T09:45:00+02:00 09:45 00:40 Confidential compute is a new compute and programming paradigm to run an application in enclave, a run-time encrypted and authenticated trusted execution environment. We give an overview of the current technologies provided by AMD, Intel and ARM. We also give an overview of open source tools to leverage compute along a tutorial to enclave any applications with few command lines. all-systems-go-2023-217-confidential-compute-state-of-the-art-and-how-to-get-started Sebastian Gajek en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/7LVG99/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/7LVG99/feedback/ Main Hall 20 min talk + 5 min Q&A 2023-09-14T10:30:00+02:00 10:30 00:25 The talk wants to provide a brief introduction into Confidential Containers Project. We'll discuss the rationale behind Confidential Computing and how concepts like Trusted Computing or Remote Attestation can be leveraged by end-users to guard their workloads not only from malicious actors but also their cloud service provider. Confidential Containers, an open-source CNCF project, aims to extend the experience of deploying cloud-native software on Kubernetes with the option to move sensitive workloads into confidential enclaves with minimal friction to the user experience. We'll introduce the components and container technologies we are using to achieve that, hint at some conceptual problems we are facing and provide a simple example of how confidential containers work in practice today. all-systems-go-2023-242-trusted-confidential-and-cloud-native-workloads-an-intro-to-the-confidential-containers-project Magnus Kulke en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/GFDUHW/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/GFDUHW/feedback/ Main Hall 20 min talk + 5 min Q&A 2023-09-14T11:00:00+02:00 11:00 00:25 openSUSE is a general purpose, rpm based distribution. One of it's unique features is the use of btrfs snapshots to offer rollback of the root file system of both traditional as well as transactional systems. This talk explains the challenges faced to integrate systemd-boot into openSUSE. all-systems-go-2023-189-systemd-boot-integration-in-opensuse Ludwig Nussel en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/E9NVZE/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/E9NVZE/feedback/ Main Hall 20 min talk + 5 min Q&A 2023-09-14T11:30:00+02:00 11:30 00:25 Modification of the kernel command line has historically been one of the easiest ways to customize system behavior. Bootloaders allow for persistent changes via config-files and on-the-fly changes interactively during system boot. System behavior changes made via the kernel command line are not limited to the kernel itself. Userspace applications from installers to init systems and beyond also take input from /proc/cmdline. It is clear that some kernel command line options are desirable (console=ttyS0 verbose) and possibly even necessary. Others, such as the cromulent 'init=/bin/sh', can allow circumvention of benefits that Secureboot and TPM provide. How to control access to kernel command line modification is a non-trivial subject. A recent pull request to systemd that added "command-line addons" garnered hundreds of comments. This talk will cover: * The stub loader 'stubby' and its allowed-list approach to kernel command line options. * Systemd-stub’s solution for command line customization * System changes that can be made through kernel command line. * Alternative channels such as smbios oem strings, or qemu 'fw_cfg' all-systems-go-2023-231-kernel-command-line-and-uki-systemd-stub-and-the-stubby-alternative Scott Moser en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/T3QFGS/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/T3QFGS/feedback/ Main Hall 20 min talk + 5 min Q&A 2023-09-14T12:00:00+02:00 12:00 00:25 This talk will explore the ideas from Lennart's "Fitting Everything Together" blog post, particularly the A/B partitioning scheme and its bootloader design, comparing it with the approach used on the SteamDeck. Spoiler alert, we're not using sd-boot. We will focus on the requirements that drove us to the latter design, some implementation details, and hurdles we needed to overcome to achieve that project. Lastly, the idea of finding common ground will be entertained where audience participation is greatly encouraged. What features would be acceptable by the wider systemd community? Would those be enough for the SteamDeck to jump ship? all-systems-go-2023-210-a-story-of-a-bootloader-w-wthree-bootloaders Emil Velikov en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/SMQPWM/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/SMQPWM/feedback/ Main Hall 20 min talk + 5 min Q&A 2023-09-14T12:30:00+02:00 12:30 00:25 A/B partitioning is great - you hermetically drop-in the whole new OS and boot into it. Although, how can we manage and migrate the RW configuration and state files that lie within? Can we do that reliably on both OS upgrades and downgrades? This talk will explore the design used on the SteamDeck, the issues we've seen while drawing analogies, and future inspiration with "Fitting Everything Together" by Lennart Poettering in mind. all-systems-go-2023-211-a-b-partitioning-let-s-talk-about-the-dirty-rw-files Emil Velikov en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/MPAEFK/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/MPAEFK/feedback/ Main Hall 35 min talk + 5 min Q&A 2023-09-14T14:30:00+02:00 14:30 00:40 systemd-repart has recently learned many features to make it useful for building discoverable disk images. In this talk, we'll give a deep-dive on the new features and how they can be used to assemble discoverable disk images. all-systems-go-2023-191-systemd-repart-building-discoverable-disk-images Daan De Meyer en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/VPQADA/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/VPQADA/feedback/ Main Hall 35 min talk + 5 min Q&A 2023-09-14T15:15:00+02:00 15:15 00:40 Recently, atomic updates via image based systems have become more relevant for servers and desktops, as they allow predictable management of large fleets. In the embedded Linux space, this approach has been the default for many years and proven updaters exist already. In this talk, we will delve into RAUC and look at how its design and features have been driven by the requirements for robust, atomic updates. The presentation will introduce the fundamental concepts surrounding A/B fallback and update signing in the context of embedded Linux updates. We will then explore the commonalities and differences between RAUC and systemd's sysupdate. The discussion will progress to cover RAUC's bundle-based update system, which allows for comprehensive system updates without the need for local storage, thanks to HTTP streaming. Additionally, we will demonstrate how adaptive updates minimize download sizes without necessitating version-specific patch management. all-systems-go-2023-213-exploring-rauc-a-flexible-building-block-for-image-based-updates Rouven Czerwinski en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/NEQ9TX/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/NEQ9TX/feedback/ Main Hall 35 min talk + 5 min Q&A 2023-09-14T16:30:00+02:00 16:30 00:40 mkosi is a tool for building operating system images. In this talk we'll give an introduction to mkosi, how we use it to develop systemd and discuss how we want to support running and updating systems with mkosi and other systemd tooling. all-systems-go-2023-190-mkosi-building-bespoke-operating-system-images Daan De Meyer en Github repository: https://github.com/systemd/mkosi/ Initial blog post on mkosi: https://0pointer.net/blog/mkosi-a-tool-for-generating-os-images.html false https://cfp.all-systems-go.io/all-systems-go-2023/talk/ASV8ZM/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/ASV8ZM/feedback/ Main Hall 20 min talk + 5 min Q&A 2023-09-14T17:15:00+02:00 17:15 00:25 BuildStream is a tool for building / integrating software stacks. In a way, it has a similar goal to bitbake / yocto and Android repo, but takes a completely different approach. It can be used to take software from various sources, build it with various buildsystems in a reproducible sandbox, and cache results for speedy rebuilds. In this talk I give a brief overview of Buildstream, how it is used to build GNOME OS, and the challenges we face in using it. I also go over freedesktop-sdk which is a base runtime that can be used as a base to build your own system. I also discuss the challenges we encountered with using buildstream with ostree and the steps we're taking to support updating with systemd-sysupdate. all-systems-go-2023-206-building-image-based-oses-with-buildstream Abderrahim KitouniValentin David en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/G8UZGL/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/G8UZGL/feedback/ Main Hall 20 min talk + 5 min Q&A 2023-09-14T17:45:00+02:00 17:45 00:25 In this talk we’ll discuss antlir2, Meta’s solution to building container and bare metal operating system images. We’ll talk about how we have built performant, hermetic and deterministic image building infrastructure on top of buck2 (Meta’s new open source build system) and how we enable users to compose their own multi-language projects with full operating systems, write tests and deploy their images. Along the way, we’ll also cover how antlir2 wrangles dnf and other upstream tooling to behave more predictably for better, more reliable images. all-systems-go-2023-223-antlir2-deterministic-image-builds-with-buck2 Vinnie Magro en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/9E9MLC/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/9E9MLC/feedback/ Main Hall Lightning talk 2023-09-14T18:25:00+02:00 18:25 00:05 Closing session of All Systems Go! 2023 all-systems-go-2023-240-closing-session-of-all-systems-go-2023 en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/PKSMVD/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/PKSMVD/feedback/ Dome 20 min talk + 5 min Q&A 2023-09-14T10:00:00+02:00 10:00 00:25 The Linux Userspace team at Meta aims to make significant contributions to upstream userspace projects, while also ensuring that Meta is able to leverage those improvements. In this talk we'll give an overview of the team and brief history of how it was formalized. Then we'll dive deeper into some of the efforts we've worked on with the open source community and features we've adopted internally. Come if you enjoy hearing about systemd, BPF, distributions, and more! all-systems-go-2023-193-adventures-of-linux-userspace-at-meta Anita Zhang en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/YAHVZG/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/YAHVZG/feedback/ Dome 20 min talk + 5 min Q&A 2023-09-14T10:30:00+02:00 10:30 00:25 The Talos Linux distribution is built from scratch with the goal of providing a secure, verified, and minimal-footprint operating system for running Kubernetes clusters. Talos is designed to be immutable, minimal, and secure. Talos includes only the bare minimum required to run Kubernetes. This talk will cover how Talos uses Unified Kernel Images (UKIs) to provide immutable, verified, and secure booting. We will also cover how Talos partially conforms to the Linux Userspace API Group specification (UAPI) to implement some of the best practices with regards to fully verifiable TrustedBoot extending to the userspace. all-systems-go-2023-202-talos-linux-trustedboot-for-a-minimal-immutable-os Noel Georgi (he/him/they/them) en With the upcoming Talos 1.5 release, Talos ships with custom ISO and metal images that are UKI compliant. This means that the kernel, initramfs, and the root filesystem are all signed and verified by the bootloader. This allows Talos to provide a fully verified boot process from the bootloader to the userspace attested by TPM. This talk will cover the following topics: - Building UKI (ukify.py implementation in Go) - Issues with reproducibility - sd-boot - sd-stub - Upgrades/Rollbacks - systemd-measure and systemd-cryptenroll partial implementation in Go Future work: - IMA attestations for userspace runtime binaries (etcd, kubelet, containerd, etc) - Talos system extensions as sd-stub compatible sysexts - Kexec with Secureboot (how can we verify the TPM PCR values are populated correctly with values from new UKI) false https://cfp.all-systems-go.io/all-systems-go-2023/talk/Q9YBUZ/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/Q9YBUZ/feedback/ Dome 20 min talk + 5 min Q&A 2023-09-14T11:00:00+02:00 11:00 00:25 This talk will be a whirlwind overview of NixOS modules and the lessons I've learned with maintaining and writing new ones. all-systems-go-2023-214-writing-your-own-nixos-modules-for-fun-and-hopefully-profit Xe Iaso en Nix modules are the core of how you organize configuration and service config, but there's a lot of "draw the rest of the owl" subtext as to how you actually go about writing them. This talk covers some best practices for how to write and organize your NixOS modules so that you can have fun and hopefully profit from it. false https://cfp.all-systems-go.io/all-systems-go-2023/talk/DP3JXQ/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/DP3JXQ/feedback/ Dome 35 min talk + 5 min Q&A 2023-09-14T11:30:00+02:00 11:30 00:40 The build system should get out of the way to let us focus on our tasks, not be distracted by slow or unreliable builds, get fast feedback on changes, and let us know what’s in the software we’re shipping to our users. But, what does it take for a build system to be really fast and reliable? What does it take to know what’s in the software? It requires aggressive parallelism and distributed caching to avoid redundant work between colleagues. And it requires complete knowledge and control of dependencies, build isolation to identify mistakes, and reproducible builds to verify results across machines and strengthen supply-chain security. In this talk you will learn how [Google’s open source build system Bazel](https://bazel.build/) and the [purely functional package manager Nix](https://nixos.org/) join forces to provide fast, correct, and reproducible builds. all-systems-go-2023-219-fast-correct-reproducible-builds-with-nix-bazel Andreas Herrmann en In this talk I will explain what we mean by correct builds, and will motivate why fast and correct builds are important and why you would care about reproducible and isolated builds. We will see how many common build systems fail to provide these desirable properties. You will be introduced to [Google’s open source build system Bazel](https://bazel.build/) and will learn how it provides fast builds, how correctness and reproducibility is relevant, and how Bazel tries to ensure correctness. But, we will also see where Bazel falls short in ensuring correctness and reproducibility. You will learn about the [purely functional package manager Nix](https://nixos.org/) and how it approaches correctness and build isolation. And we will see where Bazel has an advantage over Nix when it comes to providing fast feedback during development. I will share how you can get the best of both worlds and combine Nix and Bazel and how you can get started with these tools. But, we will also touch on potential caveats and shortcomings of this approach. false https://cfp.all-systems-go.io/all-systems-go-2023/talk/77YDZ8/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/77YDZ8/feedback/ Dome 35 min talk + 5 min Q&A 2023-09-14T12:15:00+02:00 12:15 00:40 Arch Linux has worked with its own packaging framework - Arch Linux Package Management (ALPM) - for about 20 years. This talk is about an effort to rewrite low-level components and to create specifications for related metadata files using the Rust programming language. It will cover new projects in the ALPM (https://gitlab.archlinux.org/archlinux/alpm/) group as well as several other related ones and give an outlook on future developments using the 🦀 all-systems-go-2023-207-oxidizing-the-arch-linux-packaging-infrastructure David Runge en Arch Linux (https://archlinux.org) has worked with its own packaging framework - Arch Linux Package Management (ALPM) - for about 20 years. The tooling consists mainly of scripts for package creation (e.g. `makepkg`, written in Bash) and a package manager (`pacman`, written in C). Over the last years several projects for the improvement of the packaging and package distribution ecosystem have been started. Some of which had to reinvent the wheel. This talk is about an effort to rewrite low-level components and to create specifications for related metadata files using the Rust programming language. It will cover new projects in the ALPM (https://gitlab.archlinux.org/archlinux/alpm/) group as well as several other related ones and give an outlook on future developments using the 🦀 Slides: https://pkgbuild.com/~dvzrv/presentations/all-systems-go-2023/ false https://cfp.all-systems-go.io/all-systems-go-2023/talk/3LES8Z/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/3LES8Z/feedback/ Dome 35 min talk + 5 min Q&A 2023-09-14T14:30:00+02:00 14:30 00:40 Network operating systems commonly provide a stable userspace platform for networking devices. Integration of userspace applications as well as low-level hardware support are handled by firmware build systems. Existing build systems for network operating systems display numerous limitations by either targeting only distinct types of devices, using cumbersome methodologies to add additional features or offering insufficient capabilities regarding what to include in the firmware image. In this presentation, we provide an overview of these limitations and how we mitigate them with Replica.one, an Open Source firmware builder which targets the entire networking stack. We will focus on the solution's optimization features, its capability to generate firmware for diverse classes of devices across the entire networking stack, and the flexibility to select the desired operating system between various Linux-based distributions. all-systems-go-2023-227-replica-one-a-software-defined-operating-system Jakov Petrina Trnski en The presentation targets Linux users who are interested in replacing their existing build system infrastructure with a single unified software platform. The flexibility of image-based network operating systems will allow organizations and its users to operate their existing hardware resources more efficiently and securely. Professionals who are working in the domain of firmware build systems to integrate applications and features will benefit from Replica.one’s ease of use as well as powerful image customization capabilities. The novel element of a single platform running on an entire networking infrastructure stack will be of particular interest to a wide range of organizations and companies looking to reduce their operating cost while reaping the benefits of open source community effort. false https://cfp.all-systems-go.io/all-systems-go-2023/talk/LBWXKL/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/LBWXKL/feedback/ Dome 35 min talk + 5 min Q&A 2023-09-14T15:15:00+02:00 15:15 00:40 openSUSE Aeon (formerly MicroOS Desktop) aims to be a fully fledged modern Linux Desktop leveraging as many of the latest user space innovations available including: - Immutable OS with Transactional Updates - Secure Boot - TPM Encryption - Flatpaks & OCI containers as primary application delivery This talk will introduce the distribution, highlight the adoption of some of the latest foundational user space technologies as well as share some of the pain points being faced and invite the audience to contribute to this exciting platform. all-systems-go-2023-208-opensuse-aeon-desktop-linux-finally-done-right- Richard Brown en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/RV3UZD/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/RV3UZD/feedback/ Dome 35 min talk + 5 min Q&A 2023-09-14T16:30:00+02:00 16:30 00:40 Are you using container images with hundreds of known vulnerabilities? The majority of us are using images based on the Docker official images available on the Docker Hub. This includes base images – such as Debian and Ubuntu – as well as application images such as nginx and redis. Unfortunately these images often have hundreds of known vulnerabilities due to excessively large dependency trees with out-of-date packages. This security debt can lead to unnecessary security risks and slower development cycles. Wolfi (​​https://github.com/wolfi-dev/) is a new Linux distribution optimized for building minimal, secure container images. Wolfi maintainers prioritize a rolling release model built on a rapid package update cycle, which ensures that new vulnerabilities are remediated quickly. This talk not only describes the problems that motivate Wolfi but also provides hands-on knowledge to help developers take advantage of Wolfi. By the end of the talk, developers will learn about packaging techniques with apko and melange, tools specifically designed to build Wolfi packages and turn them into minimal, low- or no-vulnerability containers. all-systems-go-2023-241-wolfi-a-secure-by-default-distro-for-curing-container-cve-chaos James StrongCarlos Tadeu Panato Junior en Key Takeaways and Highlights Popular, off-the-shelf base images and containers often have hundreds of known vulnerabilities (“CVEs”), which can, at worst, be a security risk and, at best, be a giant time suck. Wolfi is a new secure-by-default linux distribution that prioritizes rapid package updates and, by extension, fast mean time-to-remediation for known vulnerabilities. Packages in Wolfi can form the foundation of secure, minimal base images and containers, freeing developers of tedious vulnerability management tasks and increasing security for cloud-native applications. Talk Outline The Cloud-Native Application Status Quo: Bloated, Outdated, Vulnerability-Laden Images Containers 101 Show the results of running security scanners against popular Dockerhub official images Use (grype, an open source scanner) to scan golang:latest and nginx:latest. Show via command line. Show data and analysis on package counts, package staleness, vulnerability counts of official Docker Hub images Draw on six months of daily scanning results collected by presentation team Overview of Wolfi Fast package update times Fast vulnerability mean time-to-remediation Granular packages Wolfi packages are often packaged at a more granular level than their counterparts in other distributions, which allows developers to pick and choose only the components that are essential for an image, without dragging in unnecessary functionality and attack surface. Rolling release Why not alternative approaches, either other minimal images or using other distros? Google distroless Debian-based so there can be slow update times for packages Debian - Slow package updates How to build images with Wolfi packages Explain melange and building packages Example of building a package with melange Explain apko and building images Demo of building an image with apko false https://cfp.all-systems-go.io/all-systems-go-2023/talk/V9EZSS/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/V9EZSS/feedback/ Dome 20 min talk + 5 min Q&A 2023-09-14T17:15:00+02:00 17:15 00:25 A quick journey through the Azure infrastructure, specifically looking at how image-based Linux is used for Azure Boost, what it enables, what interesting security and performance features were added and where to find them upstream. all-systems-go-2023-187-microsoft-azure-boost-image-based-linux-powering-the-azure-fleet-wait-what-really-yes- Luca Boccassi en Believe it or not, today Linux is right at the core of Microsoft Azure's infrastructure, on the very nodes that run all those fancy virtual machines. Getting there was not easy, and a lot of work was needed to meet the very stringent security and performance goals that were set. We built a custom distribution, added several security features such as signed dm-verity and kernel-enforced code integrity, came up with a way to keep state alive across kexec with PMEM, and implemented the stackable Portable Services image model that ultimately became sysexts and confexts. And much more! This talk will walk through this effort, starting with a peek under the cover at the hardware that powers it and what it enables, passing through the custom OS and ending up at all the features we added to systemd and elsewhere that you all can enjoy as well. false https://cfp.all-systems-go.io/all-systems-go-2023/talk/7URRNC/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/7URRNC/feedback/ Dome 20 min talk + 5 min Q&A 2023-09-14T17:45:00+02:00 17:45 00:25 sdbusplus generates ergonomic and compile-time type-checked dbus bindings built atop sd-bus. This library is heavily used within the OpenBMC project to provide all IPC between its many userspace processes. This talk will give an overview of how OpenBMC leverages dbus, how sdbusplus facilitates its usage, as well as an introduction on our approach for asynchronous programming with C++ co-routines. all-systems-go-2023-215-asynchronous-dbus-with-c-co-routines Patrick Williams en false https://cfp.all-systems-go.io/all-systems-go-2023/talk/QUMHR3/ https://cfp.all-systems-go.io/all-systems-go-2023/talk/QUMHR3/feedback/