Running Flatcar Container Linux on Hetzner

    Hetzner Cloud is a cloud hosting provider. Flatcar Container Linux is not installable as one of the default operating system options, but you can deploy it by installing it through the rescue OS.

    These instructions require Flatcar with version 3941.1.0 or newer.

    Creating snapshots

    Snapshots in Hetzner Cloud can be used as a base image to create new servers from. While you can manually create the snapshot, this guide will demonstrate two tools to prepare the snapshots for you.

    Packer

    Building the snapshots with Packer allows you to configure the build process to your liking.

    Requirements

    Template

    Packer requires a template that describes how the snapshots should be built. Create a new file flatcar.pkr.hcl and paste in the following content:

    packer {
      required_plugins {
        hcloud = {
          source  = "github.com/hetznercloud/hcloud"
          version = "~> 1.4.0"
        }
      }
    }
    
    variable "channel" {
      type    = string
      default = "beta"
    }
    
    variable "hcloud_token" {
      type      = string
      default   = env("HCLOUD_TOKEN")
      sensitive = true
    }
    
    source "hcloud" "flatcar" {
      token = var.hcloud_token
    
      image    = "ubuntu-24.04"
      location = "fsn1"
      rescue   = "linux64"
    
      snapshot_labels = {
        os      = "flatcar"
        channel = var.channel
      }
    
      ssh_username = "root"
    }
    
    build {
      source "hcloud.flatcar" {
        name          = "x86"
        server_type   = "cx22"
        snapshot_name = "flatcar-${var.channel}-x86"
      }
    
      source "hcloud.flatcar" {
        name          = "arm"
        server_type   = "cax11"
        snapshot_name = "flatcar-${var.channel}-arm"
      }
    
      provisioner "shell" {
        inline = [
          # Download script and dependencies
          "apt-get -y install gawk",
          "curl -fsSLO --retry-delay 1 --retry 60 --retry-connrefused --retry-max-time 60 --connect-timeout 20 https://raw.githubusercontent.com/flatcar/init/flatcar-master/bin/flatcar-install",
          "chmod +x flatcar-install",
    
          # Install flatcar
          "./flatcar-install -s -o hetzner -C ${var.channel}",
        ]
      }
    }
    

    Building the snapshots

    export HCLOUD_TOKEN=<your-token>
    packer init .
    
    # This will build the snapshot for x86 (amd64-usr) and Arm (arm64-usr).
    packer build .
    

    The packer build . command takes a few minutes to complete. Afterward you can see the snapshot names and IDs:

    ==> Builds finished. The artifacts of successful builds are:
    --> hcloud.x86: A snapshot was created: 'flatcar-beta-x86' (ID: 157132241)
    --> hcloud.arm: A snapshot was created: 'flatcar-beta-arm' (ID: 157132242)
    

    You can verify these through the hcloud CLI:

    $ hcloud image list --type=snapshot --selector=os=flatcar
    ID          TYPE       NAME   DESCRIPTION        ARCHITECTURE   IMAGE SIZE
    167650172   snapshot   -      flatcar-beta-arm   arm            0.41 GB
    167650577   snapshot   -      flatcar-beta-x86   x86            0.47 GB
    

    Extended template

    If you are looking for an extended Packer template that allows some more customization, check out github.com/apricote/flatcar-packer-hcloud .

    hcloud-upload-image

    If you do not want to deal with the complexity of Packer templates, there is an alternative CLI hcloud-upload-image that does just that.

    Requirements

    Building the snapshots

    hcloud-upload-image does not know anything about Flatcar. We need to construct the URL for the image ourselves.

    export HCLOUD_TOKEN=<your-token>
    export CHANNEL=beta
    # "current" is the latest version, you can specify alternative version here (e.g 3941.1.0)
    export VERSION=current
    
    # For x86 (cx, cpx & ccx Server Types)
    hcloud-upload-image upload \
      --architecture=x86 \
      --compression=bz2 \
      --image-url=https://${CHANNEL}.release.flatcar-linux.net/amd64-usr/${VERSION}/flatcar_production_hetzner_image.bin.bz2 \
      --labels os=flatcar,flatcar-channel=${CHANNEL} \
      --description flatcar-${CHANNEL}-x86
     
    # For Arm (cax Server Types)
    hcloud-upload-image upload \
      --architecture=arm \
      --compression=bz2 \
      --image-url=https://${CHANNEL}.release.flatcar-linux.net/arm64-usr/${VERSION}/flatcar_production_hetzner_image.bin.bz2 \
      --labels os=flatcar,flatcar-channel=${CHANNEL} \
      --description flatcar-${CHANNEL}-arm
    

    Running hcloud-upload-image upload will take a few minutes to complete. If you need x86 and Arm snapshots, you can run both in parallel.

    After it completes, you should see the following output:

    Successfully uploaded the image! image=167673693
    

    You can verify this through the hcloud CLI:

    $ hcloud image list --type=snapshot --selector=os=flatcar
    ID          TYPE       NAME   DESCRIPTION        ARCHITECTURE   IMAGE SIZE
    167673693   snapshot   -      flatcar-beta-x86   x86            0.47 GB
    167673694   snapshot   -      flatcar-beta-arm   arm            0.41 GB
    

    Creating servers

    Requirements

    Make sure that your SSH Key is available in the current Hetzner Cloud project:

    hcloud ssh-key list
    
    # If not, you can upload the public key:
    hcloud ssh-key create --public-key-from-file ~/.ssh/<your-ssh-key>.pub --name my-ssh-key
    

    Server configuration

    Flatcar allows you to configure machine parameters, launch systemd units on startup and more via Butane Configs . These configs are then transpiled into Ignition JSON configs and given to booting machines. We’re going to provide our Butane Config to Hetzner via the user-data flag.

    The coreos-metadata.service saves metadata variables to /run/metadata/flatcar. Systemd units can use them with EnvironmentFile=/run/metadata/flatcar in the [Service] section when setting Requires=coreos-metadata.service and After=coreos-metadata.service in the [Unit] section.

    As an example, this Butane YAML config will start an nginx Docker container and display the instance hostname:

    variant: flatcar
    version: 1.0.0
    
    storage:
      directories:
        - path: /var/www
    systemd:
      units:
        - name: nginx.service
          enabled: true
          contents: |
            [Unit]
            Description=NGINX example
            After=docker.service coreos-metadata.service
            Requires=docker.service coreos-metadata.service
            [Service]
            EnvironmentFile=/run/metadata/flatcar
            TimeoutStartSec=0
            ExecStartPre=-/usr/bin/docker rm --force nginx1
            ExecStartPre=-/usr/bin/bash -c "echo \"Hello from ${COREOS_HETZNER_HOSTNAME}\" > /var/www/index.html"
            ExecStart=/usr/bin/docker run --name nginx1 --volume "/var/www:/usr/share/nginx/html:ro" --pull always --log-driver=journald --net host docker.io/nginx:1
            ExecStop=/usr/bin/docker stop nginx1
            Restart=always
            RestartSec=5s
            [Install]
            WantedBy=multi-user.target
    

    Before we can create the server, we need to transpile this Butane configuration to the Ignition format:

    docker run --rm -i quay.io/coreos/butane:latest < nginx-example.yaml > nginx-example.json
    

    Create the server

    Now that we have the snapshots, SSH Key and our Ignition config, we can finally create the first server:

    # Get ID of the most recent flatcar snapshot for x86
    SNAPSHOT_ID=$(hcloud image list --type=snapshot --selector=os=flatcar --architecture=x86 -o=columns=id -o noheader --sort=created:desc | head -n1)
    
    hcloud server create \
      --name flatcar-test \
      --type cpx11 \
      --image ${SNAPSHOT_ID} \
      --ssh-key <your ssh key name or id> \
      --user-data-from-file nginx-example.json
    

    This will also take a minute or two to load the snapshot. After the process is finished, you will see the following output:

    Server 48081481 created
    IPv4: 37.27.83.94
    IPv6: 2a01:4f9:c012:52f1::1
    IPv6 Network: 2a01:4f9:c012:52f1::/64
    

    To verify that nginx was properly started, run curl $(hcloud server ip flatcar-test).

    You can log in via ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null core@$(hcloud server ip flatcar-test).

    Known limitations

    These Hetzner Cloud feature do not work with Flatcar:

    • Volume Automount: You need to mount volumes manually.
    • Setting & Resetting Root Passwords: You need to configure an SSH Key through the API or Ignition User Data.