All Systems Go! 2026

A welcome session for All Systems Go!

In recent systemd releases we substantially improved the tooling for provisioning and deploying operating systems with systemd. For example, systemd-sysinstall, systemd-sysupdate, credentials, systemd-firstboot, systemd-stub's parameterization logic, systemd-confext, systemd-imds, systemd-import all are relevant components for getting a node up and running with the right configuration, securely and robustly.

In this talk I'd like to explain the current landscape, and how the various components matter in various scenarios, and how they fit together. I'll also discuss what's next, what's missing, and where we should be going with all this.

sysupdate provides a lot of incredibly useful primitives for doing system updates, but it is still mostly designed around old-school SHA256SUMS distribution and static transfer definitions. However, the need to have more autonomous systems for updating systems and adding components became quite clear when we started developing such systems at Amutable. This approach also dove-tailed nicely into developing an autonomous provisioning system using the same underlying mechanism.

This talk will contain a detailed description of the architecture we came up with, along with some of the caveats that such an architecture has with generic distributions.

IETF is standardizing a new ACME challenge, device-attest-01, which allows organizations to provision device bound certificates to machines, and enables machines to present signing certificates that can't be extracted out of machines they where intended for. This is useful for cases where you want to provide reverse proxies with mTLS with a strong sense of device identity.

attezt is intended to be a suite of tools to work the new device-attest-01 ACME challenges for Linux. It provides an ACME client, an attestation server with (simple) support for inventory systems and a PKCS11 agent which together enables the support of this ACME challenge on Linux.

This talk will give an introduction to the new ACME challenge, a quick rundown of how an attestation server works and how attezt works.

https://datatracker.ietf.org/doc/draft-ietf-acme-device-attest/
https://github.com/Foxboron/attezt

Containers should follow an opinionated runtime model. They are designed to be immutable, minimal and predictably constrained to specific behaviors. In practice, Linux gives us powerful primitives such as namespaces and cgroups, but those mechanisms do not directly enforce the higher-level semantics that make containers attractive in the first place.

A container may be packaged as a minimal immutable image, yet still drift at runtime, execute unexpected code, load unexpected objects, or reach kernel functionality unrelated to its purpose. The result is a gap between the way containers are intended to behave and what the kernel actually enforces.

This talk presents Micromize, an open-source project that uses eBPF LSM to narrow that gap. Micromize explores how an opinionated container runtime model can be translated into kernel-enforced constraints that preserve workload behavior after deployment.

We will cover the design, implementation tradeoffs, and open questions behind enforcing stricter container semantics at runtime: what can be enforced cleanly, what remains hard, where existing Linux mechanisms help, and where eBPF LSM gives us a new enforcement point.

User space based Full Disk Encryption in openSUSE is old news. We already can set up FDE installations using the systemd toolset. This allows us to use traditional passwords, the TPM2 to validate the status before mounting the encrypted device or a FIDO2 key to validate the presence of a token owned by authorized users. To increase the security, in Tumbleweed, we recommend combining both TPM2 and a password (TPM2+PIN). This guarantees the system is in a healthy state and the user has the correct authorization, but it still doesn't support a combination of TPM2 and FIDO2 keys. To fix this issue, we are proposing a new enrollment method in systemd: TPM2+FIDO2.

The image based Linux vision is clear: immutable system images, declarative composition, no imperative package installation at runtime. But what happens when you need dozens of independently deployable application modules on the same device, each owned by a different team in a large enterprise, each on its own release cadence, while keeping the root filesystem untouched?

We built a modular deployment system for production Linux devices that takes the image based philosophy to its logical conclusion: every module is a read only ext4 filesystem image, built deterministically at build time, deployed to its own A/B partition with zero installation, and managed entirely through systemd transient services. No unit files on the rootfs. No package manager on the device. No post deploy scripts. The image is the truth.

On the packaging side, we use deterministic ext4 creation with constant timestamps to produce (mostly) reproducible images regardless of build environment. This reproducibility directly enables efficient delta based OTA: because the on device image is read only and byte identical to what was built, we can chunk-diff against it and deliver only changed blocks critical for constrained networks.

On the runtime side, a oneshot service mounts each module and starts it via systemd-run as a transient service inheriting automatic restart, cgroup resource control, and clean lifecycle semantics without ever writing a .service file to disk. We deliberately rejected the alternative of modules shipping their own unit files precisely because it violates the principle that deployment must not modify the root filesystem. Transient services give us the systemd machinery we need while preserving an immutable rootfs compatible with dm-verity.

Modules are fully isolated: separate uid/gid, sandboxed filesystem access, dedicated persistent storage, and communication exclusively through a formal interface (gRPC unix domain sockets in our case). Each module treats every other module as a remote server: no hard startup dependencies, no shared state, no assumption that any other module is even present on the same device

Same as every year, a lot has happened in the systemd project since last year's
ASG. We released multiple versions, packed with new components and features.
This talk will provide an overview of these changes, commenting on successes and
challenges, and a sneak peak at what lies ahead.

The Yocto project contains multitudes, and as such it contains the ability to create a modern, image based, integrator friendly operating system, taking advantage of various layers of tooling for hardware and feature support. Moonforge is the result of the work done by Igalia in this space, and it focuses on making it easy to create, build, test, deploy, and update a OS image for multiple products and verticals.

Let's have an open discussion with systemd developers who are at ASG and users in the audience. We will open with the developers saying what they plan to work on in the near future, and then allow questions / comments from the audience.

Recent years have spurred much discussion in Linux communities about the idea of building an OS that is both Trusted and Immutable. While successful in many ways, attempts to implement these ideas have created numerous sharp edges for end users. At the same time, these initiatives bear many advantages for the ecosystem's push towards becoming more reliable, accessible, and consumable. What lessons can we take from other OS vendors to improve user experience and accelerate adoption?

This talk will present the newest updates and advancements in NixOS related to Secure & Measured Boot. It will take a look at what we have achieved this far, what issues we have come across, what we would like to see from the ecosystem, and what we are working towards next. Additionally, it will go over some other exciting things that happened related to systemd in NixOS.

Key advancements:
- Support for Measured Boot via systemd-pcrlock in Lanzaboote (the boot security toolkit for NixOS)
- Support for Boot Counting in Lanzabooote
- Removed most patches from systemd (18 -> 6) and added a patch policy that prohibits most new patches.
- systemd initrd is now the default

Linux doesn't have a unified way to be installed. Every distribution implements their on installer with a different feature set and UX. systemd-sysinstall is a tool to install an OS to a device. It leverages repart to format the disk and bootctl to manage UEFI and the boot partition and supports disk encryption and TPM integration. In addition, it exposes a varlink interface to be used by graphical installers.

In this talk we will explore what sysinstall does, how the GNOME OS installer (gnome-setup) leverages it to install GNOME OS, why distributions shipping GNOME should switch to gnome-setup, and how other desktop environments can build installers using the same stack.

sdme is a single static binary that boots systemd-nspawn containers using copy-on-write roots (overlayfs layers or btrfs snapshots), with OCI registry pulling and Kubernetes Pod YAML support. No daemon, no runtime dependency beyond the systemd you already run.

Why I built it. Every container runtime reinvents what systemd already does well: process supervision, logging, cgroups, service ordering, socket activation. sdme makes the opposite bet. systemd-nspawn already boots a full system in isolation; the kernel already gives it a cheap copy-on-write root, whether through overlayfs or a btrfs snapshot. Put a thin, daemonless CLI over the two and you get container workflows (clone, import, pull, run) without a new runtime to install or a daemon to keep alive. It began as a one-command clone of the running host, but the bet is bigger than that: lean on the init system instead of rebuilding it.

What you get. Proper machine-like containers: each one runs a full systemd init with journald, D-Bus, and systemctl, not a single foreground process. Each container is highly configurable through native systemd surfaces, with per-container resource limits, networking, and drop-in config. Clone your running host into a throwaway container in one command; run a Docker Hub image (nginx, redis, postgres) as a managed systemd service instead of a foreground process; deploy a multi-container pod from Kubernetes Pod YAML with health probes, secrets, and configmaps. A rootfs can also come from any distro, an OCI registry, a tarball, or a QCOW2 image, and you can build your own from a Dockerfile-like config.

How it works. One copy-on-write root and one systemd-nspawn invocation give you a fully booted system. Storage is pluggable per container. The overlayfs backend mounts the host / (or any imported rootfs) as an immutable lower layer with a fresh upper, so clones are instant. The btrfs backend snapshots the base rootfs as a subvolume instead, which makes the container root a real filesystem: nested containers work, user namespaces use native idmapped mounts, and per-container disk quotas come from qgroups. It runs directly on a btrfs data directory, or inside a loopback pool image on any other filesystem. The binary does two jobs: it manages those copy-on-write roots, and it drives systemd over D-Bus to start, stop, and supervise containers. Everything else (init, journald, cgroups, service ordering) is systemd doing what it already does, with no daemon in the middle.

For a while, the standard way to run Podman containers under systemd was podman generate systemd. It worked, but the output was static and awkward to maintain. Quadlet replaced it — and most guides stop at "drop an INI file in a directory and your container starts." leaving the machinery a black box.

This talk demystifies that black box, because the internals are exactly what you reach for when things break. Quadlet isn't a daemon. It's a systemd generator that runs at daemon-reload and turns your .container, .pod, and .network files into ordinary .service units.

In session, we'll trace one stack example from end to end, watching a three-line container file expand into a full podman run command, and cross-unit references become injected After=/Wants= dependency edges with name-mangled volumes and networks.

You'll leave knowing exactly what gets generated, why edits need a daemon-reload, how to dump & debug a failing unit yourself, and explain why keeping only the source files under version control is reliable. Less magic, more model.

Systemd can now be linked as a single static binary, which is enough to run a "full system" container that boots into a running manager. Transient units can then be used to do more things in the container. Thanks to systemd-nsresourced and systemd-mountfsd, this now works fully unprivileged and with no suid helpers. I want to show that this actually works and can be used for experimentation, development, and custom container images.

Using mkosi and apparmor.d as building blocks, this talk presents a prototype minimal Debian server. The goal is not to provide yet another Linux distribution, but to explore how hard it is to build personalized, hardened server images.

On top of the usual mkosi-backed features such as UKIs, A/B updates and dm-verity, the project adds a whole-system AppArmor policy and an immutable /etc. Because older devices also matter, the prototype supports the Raspberry Pi 3 (UEFI) too, with minor changes to the security model. The talk presents the differences in design, attacker model, and implementation between the various targets.

Recently systemd acquired functionality to download & invoke OCI containers directly. In this talk I'd like to explain where we are at with this, what we have, where the gaps are and where we are going, to make OCI containers a thing you can directly execute with systemd, without external tools.

Immutable, image-based Linux has gone mainstream — on Fedora. Nearly every atomic distro descends from the RPM/ostree lineage. Debian is conspicuously rare, with really only Vanilla OS and Endless OS to its name. There's a reason, and it's apt: Debian's package manager expects a writable root filesystem, so the moment you seal the rootfs, the native way to install and update software stops working. That's the wall every immutable Debian hits, and most climb it with Flatpak or containers. This talk takes a different route — Nix. Nix keeps everything in a self-contained, content-addressed /nix/store, so it never needs a writable rootfs. The base is built once with mkosi and frozen (apt isn't even installed) and Nix manages everything that changes after that, reproducibly. The result is an immutable Debian rootfs with the entire Nix package ecosystem available on top.

The storage directory settings in systemd help define where services store their data. Two important features have been implemented for these directories. The first one is id-mapped mounts, which is a filesystem feature that allows a mount namespace to show a different UID than what is stored on a file. Storage directories now support id-mapping, so that the files within the mount namespace of a service defined with DynamicUser=yes are owned by its unprivileged UID/GID. The second feature is storage quota support. Storage limits can now be defined in terms of percentages or absolute values to enforce quotas on the consumption of State, Cache, and Logs directories. These features enhance the security and resource management of systemd services.

The limitations of systemd-sysext/confext from the early days are mostly gone nowadays. The loading of extension images from the initrd means that generic content will behave as expected without any workarounds. New settings help making live reconfiguration work better as well. We will see how declarative configuration management with systemd-sysext/confext from git can be done with an OpenTofu/Terraform module. For image-based Linux binding configuration together with a pinned OS version is now possible through bootctl link. OS builders can ship /etc files through a default confext image.

System extensions (sysexts) become part of the host's /usr, making them available to all users. But what if we want to run tightly coupled services without exposing their binaries to everyone else on the system?

This lightning talk will show how to ship a real-world stack (kubelet, containerd, and runc) without making them accessible to the rest of the host. We will demonstrate how to use portable services and systemd capsules to allow these components to invoke each other and share some namespaces, all while keeping the host OS clean.

Observability stacks have gotten good at CPU, memory, and network — but storage is still where the picture goes dark. When a workload stalls on I/O, device-level tools like iostat or blktrace tell you the disk was busy; they don't tell you which service caused it, or where in the stack the latency actually accrued: a slow fsync, writeback pressure in the page cache, a filesystem journal commit, or a saturated block queue.

This talk follows a single write from the syscall boundary down to the block layer, using eBPF to instrument each handoff — VFS, the filesystem, the page cache, and block I/O submission and completion — without patching applications or running a custom kernel. We'll look at which attach points (tracepoints, fentry/fexit, kprobes) give honest per-layer latency, how to keep the measurement overhead from biasing the result, and how to attribute the latency back to the responsible cgroup or systemd service instead of an aggregate line on a dashboard. Examples range from fsync-heavy database commits to large sequential writes that look fine at the device level but stall well above it.

Attendees will leave with a concrete method for turning storage from a black box into a traceable path — and an honest sense of where eBPF helps, and where it still can't see.

When you lock your systemd-homed enabled Linux desktop today, your home directory (including the desktop session) is fully frozen. This happens because systemd-homed uses a single cryptographic key (the LUKS volume key) and throws it away during lock. Want to lock away just your credentials as on a normal screen lock while keeping music playing and notifications coming through? You can't.

This talk presents the work done as part of the systemd STF grant to extend the systemd-homed's key hierarchy that fixes this issue.

Marco will review the state of systemd sandboxing in Debian packages, what we need to fix to have more and what will break by enabling too much of it.

In this talk I'd like to provide a vision for accessing systemd hosts via local IPC APIs and remote RPC APIs. With the Varlinkification of systemd going full steam ahead, I'd like to explain what I think should be next for systemd and how things fit together with our focus on APIs.

We'll talk about metrics/reports, Varlink in general, the Varlink HTTP proxy specifically, about security models, and zero-agent remote access, secure system parameterization and more. We'll talk about design patterns to follow, both for systemd components, and for other OS components, outside of it.

KDE's new image-based Linux distribution embraces modern technologies whenever possible. We'll explore all the tools that contribute, and how they are being utilized to form the final product. From building the disk image to eventually updating it on the user's system.

Enterprise Linux deployments lack a standardized, native enrollment flow. Current solutions rely on distro‑specific first‑boot mechanisms (systemd-firstboot, cloud‑init, realmd, ignition, kickstart and more) or external configuration management layered post‑deployment, resulting in poor feature sets w.r.t. modern features enabled by TPM2 or UEFI setting management.

This talk proposes to explore minimal, vendor‑neutral set of APIs and protocols for OS enrollment, analogous to Windows Autopilot but designed for the Linux userspace stack and specifically for the ParticleOS ecosystem. We will examine a concrete model based on device identity (TPM, SMBIOS information), secure attestation, policy discovery, and management‑hint delivery, decoupled from any particular configuration management system.

There might be a live demonstration presenting a reference implementation on NixOS communicating with a lightweight enrollment service over HTTP(S) leveraging a bunch of Varlink APIs built across the past years and presented at ASG.

The primary outcome is a community‑reviewed specification draft for enrollment APIs, intended for integration into systemd and distro installers.

The talk is not a proposal for a new tool but rather invite the attendees to think about the problem space around corporate enrollment for Linux distributions and how do we build native solutions for distributions interested by this idea.

Confidential VM (CVM) instance types are becoming ubiquitous, with all major cloud providers offering them as a key component for achieving “zero trust.” While complete zero trust is currently more of a marketing term than a technical reality, we can acknowledge that CVM technologies help reduce the Trusted Computing Base (TCB) when utilized properly. But do we actually know how to use them effectively?
When transitioning from a world of traditional, mutable operating systems, users often expect to gain security benefits simply by moving their existing workloads to new CVM instance types, perhaps with some level of attestation. Unfortunately, in many cases, this looks more like a role play in a “security theater”.
Over the past few years, we have investigated what security benefits can truly be unlocked when mutable RHEL or Fedora workloads are deployed on confidential VMs, whether in the public cloud or on-premises. While immutable OSes offer distinct advantages in such scenarios, we pose the question: can traditional (mutable) OSes also benefit? Over the last year, we have contributed several features to systemd (and other components) that we believe can improve the situation. In particular, we are focused on preventing certain attack scenarios against a VM which is normally deployed from a publicly available cloud (marketplace) image. We have further ideas, and in this talk, we will share our vision for how these pieces can fit together.

For years, kexec was the standard way to skip lengthy firmware and bootloader delays at reboot. However, in modern cloud environments, even a fast reboot causes severe service degradation. Guests evacuation in virtualized environments or rebuild of massive in-memory caches for large databases is costly and disruptive.

Kernel HandOver (KHO) and Live Update Orchestrator (LUO) introduce a complete framework for seamless kernel and userspace state preservation across upgrades.

KHO provides the infrastructure to preserve kernel data structures and arbitrary memory regions across kexec. Physically contiguous memory areas managed by KHO let the incoming kernel to bootstrap itself without touching the preserved memory.

On top of KHO, LUO provides userspace APIs for coordination of the transition and allows applications to preserve file descriptors across kexec. With initial support for memory file descriptors (memfd), the ecosystem is rapidly expanding to cover subsystems like PCI, VFIO, KVM, guestmemfd, and iommufd.

Moreover, the ecosystem now features native systemd integration, leveraging the standard fdstore interface to pass resources across the kernel reboot.

When a VM is confidential (CVM), the host cannot access its memory, enabling data-in-use protection. Hardware vendors like Intel (TDX) and AMD (SEV-SNP) provide the underlying technology to achieve this. The use case becomes more complex when a persistent disk is added to a CVM: the disk must also be encrypted, but how can we safely manage its encryption keys without trusting the host? We propose a mechanism to encrypt and decrypt a disk using remote attestation: the passphrase is stored in a Key Broker Service (KBS) that only releases it when the identity of the VM is verified. During the first boot, systemd-repart encrypts the root disk using a random key that is generated and stored in KBS. On subsequent boots, systemd-cryptsetup requests the same key to decrypt the disk. We present the changes we are working on to support this use case in components like KubeVirt, Trustee, and systemd-repart. We present PRs for Trustee, systemd-repart, and KubeVirt, along with a new LUKS token plugin that integrates systemd-repart and systemd-cryptsetup with KBS for key retrieval. This use case extends beyond KubeVirt and applies to any environment where confidential workloads require encrypted persistent storage, including bare metal deployments.

When a machine/server/node is serviced and experiences a downtime, that results in a service interruption, customers or users tend to get quite sad. This talk will explore two recent additions to systemd that aim to make these service interruptions as minimal as possible.

IncusOS is an immutable OS based on Debian 13 and built using mkosi. It's designed to be as secure as possible, actively relying on both UEFI Secure Boot and a TPM 2.0 module to provide strong boot security as well as encryption at rest through LUKS+TPM.

Its whole purpose is to provide an ideal environment for running Incus with a focus on running on bare metal with pretty complex local and remote storage as well as a variety of networking options.

Unlike most other Linux systems, IncusOS doesn't provide any local or remote shell access to the system. Instead it's entirely API driven with its management API being available through the Incus API itself.

That management API allows for system configuration, update management, installation of additional components (systemd system extensions), configuring a variety of system services, ... But it also needs to handle a variety of fallback and recovery/patching mechanisms for when something goes wrong.

This talk will cover why we built IncusOS, how we built it, its current state and what we're working towards in the near future.

BuildStream is a tool for building / integrating software stacks. In a way, it has a similar goal to bitbake / yocto and Android's repo, but takes a completely different approach. It can be used to take software from various sources, build it with various buildsystems in a reproducible sandbox, and cache results for speedy rebuilds.

mkosi is a tool for easily building customized OS images. It is generally used with distros that have a package manager such as dnf or apt. There is however recent development to add support for using it with BuildStream.

In this talk I give a brief overview of Buildstream, and freedesktop-sdk which is a BuildStream project that can be used as a base to build your own operating system. While BuildStream on its own can build the whole thing, there are some advantages to using mkosi which I'll go over in this talk.

I'll also go over the challenges faced moving GNOME OS to use mkosi for generating the image.

Systemd provides a growing set of very useful varlink APIs. With the new Rust-based varlink-http-bridge, it is possible to expose any number of them over the network securely via https/websockets, including over vsock, with all of the interesting varlink features like "more" support or "protocol-upgrade" support.

This presentation gives an overview of why varlink and websockets are such a good fit, why the varlink-http-bridge implementation is mostly just a transparent byte proxy, and how it plugs into varlinkctl seamlessly to make networked varlinkctl calls "just work". It also explains the security model and the plans for the future.

The project lives in https://github.com/systemd/varlink-http-bridge.

Delivering a Linux OS via OCI images unifies infrastructure management but often introduces severe network and state-reconciliation bottlenecks. This talk dives into the mechanics of solving these challenges by applying OS updates directly from OCI images to btrfs snapshots. We will explore how to drastically optimize these atomic upgrades using zstd-chunked compression, enabling systems to execute file-level delta updates via HTTP range requests instead of pulling full layers. Finally, we will demonstrate how to leverage snapper for seamless 3-way merges, safely reconciling upstream OS changes with local user configurations.

Image-based Linux systems have different storage requirements from traditional mutable systems as they rely on signed, immutable disk images for software delivery and separate writable filesystems for data.

Technologies like systemd-homed require that multiple writable filesystems share the same backing storage. This thin provisioning is not something that is well supported by Linux today as filesystems are not aware of it. Another area for improvement is that many disk images often contain similar data that could be deduplicated.

We will present a storage system prototype designed for the usecases of image based systems and our plans for improving thin provisioning on Linux.

Varlink is slowly getting adopted in the desktop space. This brings a few new challenges, one of them being observability: D-Bus allows us to capture the entire system and session bus and then drill down on specific interactions in the system. Varlink on the other hand is just a protocol spoken over AF_UNIX sockets and thus has no bus and no global view of the system. The talk explores how we can use eBPF and extended attributes on sockets to give us back the observability we're used to with D-Bus.

In this talk, we'll walk through the architecture of Amutable's OS image build system. We'll dive into how we mirror rpms from Fedora and orchestrate building those rpms from source, as well as building OS images with buck2, an open source monorepo build system.

This talk will give an update on make NoNewPrivs the default on a Linux distribution.

A fundamental aspect of an OS security policy is the ability to manage mounting. That includes filesystem mounts, remounts, bind-mounts, and property changes. Contrary to popular believe Linux exposes only barebones policy hooks. This is about to change.

A decade ago, container networking was in its infancy, relying heavily on complex bridges and iptables rules. Today, the landscape has fundamentally shifted towards highly performant, secure, and observable dataplanes powered by eBPF.

In this talk, a core contributor to Cilium reflects on a decade of building container networking solutions - from the early days of Weave Net to modern eBPF-based architectures. We will explore the hard-fought engineering lessons from migrating away from traditional kernel bottlenecks and discuss how Cilium evolved into the most widely used container networking plugin (CNI) and a foundational platform for advanced networking research. Finally, we will look to the horizon to discuss what the next decade of Linux system networking holds.

On a standard Linux OS it is not possible to implement a policy for onlining block devices. Newer kernels will rectify this. Let's figure out how this works.

In this talk we'll cover the developments in kmod over the past few years. We'll wiz past the documentation and contribution updates, outline some of the performance improvements and discuss the latest APIs introduced.

Closing session of All Systems Go! 2026