🌬️ Installing Airflow¶

Apache Airflow is an open-source workflow orchestrator: workflows are defined as Python DAGs (directed acyclic graphs) whose tasks are scheduled, executed, retried and monitored from a web UI.

This page installs a self-contained Airflow instance on this server, running rootless under Podman with systemd Quadlet units and a PostgreSQL metadata database.

Features:

🐍 workflows defined as Python DAGs, scheduled and monitored from a web UI
🗄️ LocalExecutor backed by a dedicated PostgreSQL metadata database
🧩 one container per component (scheduler, dag-processor, api-server) — no always-on triggerer
🛡️ runs in its own isolated rootless environment with a dedicated Linux user
⚙️ idle-CPU tuning suited to a low-traffic instance on an ARM board

Info

The project is open-source and can be downloaded here: https://airflow.apache.org.

📥 Installation¶

This section describes how to install and run Podman services using systemd Quadlet, enabling containers to restart automatically. It allows the service to run in its own isolated rootless environment with a dedicated Linux user.

📋 Requirements¶

Info

Airflow requires the installation of

NGINX
Podman
podmanctl (to start/stop/update the service stack)

⚙️ Configuration¶

📋 Service Setup¶

Info

Please follow the following installation steps:

Define Service Variables

# define the service name
SERVICE_NAME=airflow

Initialize Service Environment

# built-in bash safety and SERVICE_NAME validation
set -euo pipefail; [ -n "${SERVICE_NAME:-}" ] || { echo "SERVICE_NAME is empty"; exit 1; }

# automatically generate variables
SERVICE_HOME="/media/ssd/podman-users/${SERVICE_NAME}"
SERVICE_DIR="/media/ssd/podman/${SERVICE_NAME}"
SERVICE_USER="${SERVICE_NAME}_svc"
SERVICE_ADMIN="${SERVICE_NAME}_admins"

🔐 Service Isolation¶

Configure Rootless Podman UID/GID Namespace Mappings

# allocate rootless Podman UID/GID mappings if missing
SUBID_SIZE=65536
allocate_subids_if_missing() {
  local user="$1"
  local start
  if ! grep -q "^${user}:" /etc/subuid; then
    start="$(awk -F: 'BEGIN { max = 100000 } { end = $2 + $3; if (end > max) max = end } END { print max }' /etc/subuid)"
    sudo usermod --add-subuids "${start}-$((start + SUBID_SIZE - 1))" "$user"
  fi
  if ! grep -q "^${user}:" /etc/subgid; then
    start="$(awk -F: 'BEGIN { max = 100000 } { end = $2 + $3; if (end > max) max = end } END { print max }' /etc/subgid)"
    sudo usermod --add-subgids "${start}-$((start + SUBID_SIZE - 1))" "$user"
  fi
}
allocate_subids_if_missing "${SERVICE_USER}"

# verify rootless Podman UID/GID namespace mappings exist
grep -q "^${SERVICE_USER}:" /etc/subuid || { echo "ERROR: missing subuid mappings for ${SERVICE_USER}"; exit 1; }
grep -q "^${SERVICE_USER}:" /etc/subgid || { echo "ERROR: missing subgid mappings for ${SERVICE_USER}"; exit 1; }

# validate that all subordinate ID ranges in a file are non-overlapping
validate_subid_ranges() {
  local file="$1"
  awk -F: '
    {
      start = $2
      end   = $2 + $3 - 1
      for (i = 1; i <= count; i++) {
        if (start <= ends[i] && end >= starts[i]) {
          printf "ERROR: overlapping ranges in %s\n", FILENAME
          printf "  %s:%s:%s overlaps %s:%s:%s\n",
            $1, $2, $3,
            names[i], starts[i], sizes[i]
          exit 1
        }
      }
      count++
      names[count]  = $1
      starts[count] = start
      ends[count]   = end
      sizes[count]  = $3
    }
  ' "$file"
}
validate_subid_ranges /etc/subuid
validate_subid_ranges /etc/subgid

# configure the rootless storage driver BEFORE Podman first initializes its storage.
# fuse-overlayfs is a reliable rootless overlay backend and avoids kernel-version
# quirks of native rootless overlay on ARM/Armbian. It is optional with the default
# userns used here, but recommended; set it before the first storage init so no
# 'podman system reset' is needed later.
sudo -u "${SERVICE_USER}" mkdir -p "${SERVICE_HOME}/.config/containers"
sudo -u "${SERVICE_USER}" tee "${SERVICE_HOME}/.config/containers/storage.conf" >/dev/null <<'EOF'
[storage]
driver = "overlay"

[storage.options.overlay]
mount_program = "/usr/bin/fuse-overlayfs"
EOF

# rebuild rootless Podman state to ensure existing containers/storage use current namespace mappings
# use a login shell (-i) so HOME points at the service user's home
# (storage now initializes with fuse-overlayfs; no 'podman system reset' needed)
sudo -iu "${SERVICE_USER}" podman system migrate

Configure Secure Service Directory Permissions

# Apply the admin/service ACL policy to one directory tree, deterministically.
#   $1 = tree to treat
#   $2 = (optional) a sub-path to skip entirely (e.g. a container data subtree)
#
# Why setfacl, not chmod: on an ACL'd tree, chmod edits only the mask, never group::,
# so stale execute bits survive. setfacl sets every entry explicitly (mask included),
# lowercase — the result is reproducible and no execute bit can resurface.
apply_acl_tree() {
    local tree="$1"
    local skip="${2:-}"
    local prune=()
    [ -n "$skip" ] && prune=( -path "$skip" -prune -o )                                         # skip this subtree if a 2nd arg is given

    sudo find "$tree" "${prune[@]}" -exec chown "${SERVICE_USER}:${SERVICE_ADMIN}" {} +         # service user owns; admin group is the owning group

    # directories — access ACL (perms on the existing dirs)
    sudo find "$tree" "${prune[@]}" -type d -exec setfacl    -m g:${SERVICE_ADMIN}:rwx {} +     # admin group: access existing dirs
    sudo find "$tree" "${prune[@]}" -type d -exec setfacl    -m u:${SERVICE_USER}:rwx {} +      # service user: access existing dirs
    sudo find "$tree" "${prune[@]}" -type d -exec setfacl    -m u::rwx,g::rwx,o::-,m::rwx {} +  # owner/group rwx, deny others, pin mask rwx

    # directories — default ACL (inherited by NEW files/dirs created later)
    sudo find "$tree" "${prune[@]}" -type d -exec setfacl -d -m g:${SERVICE_ADMIN}:rwx {} +     # admin group: inherit access on new items
    sudo find "$tree" "${prune[@]}" -type d -exec setfacl -d -m u:${SERVICE_USER}:rwx {} +      # service user: inherit access on new items
    sudo find "$tree" "${prune[@]}" -type d -exec setfacl -d -m u::rwx,g::rwx,o::- {} +         # default owner/group/other for new items

    # files — access ACL (rw only, never executable)
    sudo find "$tree" "${prune[@]}" -type f -exec setfacl    -m g:${SERVICE_ADMIN}:rw {} +      # admin group: access existing files
    sudo find "$tree" "${prune[@]}" -type f -exec setfacl    -m u:${SERVICE_USER}:rw {} +       # service user: access existing files
    sudo find "$tree" "${prune[@]}" -type f -exec setfacl    -m u::rw,g::rw,o::-,m::rw {} +     # owner/group rw, deny others, pin mask rw

    sudo find "$tree" "${prune[@]}" -type d -exec chmod g+s {} +                                # setgid: new items inherit the group (a MODE bit; ACLs can't set it)
}

# --- SERVICE_HOME: Podman's private runtime home ------------------------------
# Own the home + set setgid, then ACL ONLY the .config subtree (containers.conf,
# storage.conf, and the *.network / *.container quadlets you hand-edit) so admins
# can edit them sudo-less.
#
# Leave ~/.local (storage) and ~/.cache to Podman: ACLs on the overlay store break
# the runtime ("OCI permission denied" on the `merged` mount). Admin reaches them via sudo.
sudo chown "${SERVICE_USER}:${SERVICE_ADMIN}" "${SERVICE_HOME}"                       # service user owns; admin group is the owning group
sudo chmod u=rwx,g=rwx,o=,g+s "${SERVICE_HOME}"                                       # owner/admin access; deny others; inherit group on new items
apply_acl_tree "${SERVICE_HOME}/.config"                                              # .config subtree: full sudo-less admin ACL treatment

# --- bind-mount data dirs: created + owned BEFORE the SERVICE_DIR ACL pass ---------
# Each container writes as a mapped subuid, so its data dir must be owned by that subuid
# (chown as root; only root crosses the id range). We create + own them FIRST so they
# neither inherit SERVICE_DIR's default ACL (inheritance happens at creation) nor get
# walked by apply_acl_tree (which prunes data/).
#
# No ACLs here: Postgres refuses a group-accessible PGDATA. Admin reaches them via sudo.
SUBUID_BASE="$(awk -F: -v u="${SERVICE_USER}" '$1==u {print $2}' /etc/subuid)"
SUBGID_BASE="$(awk -F: -v u="${SERVICE_USER}" '$1==u {print $2}' /etc/subgid)"

# data subdir : in-container uid/gid it must be owned by
data_dirs=(
    "airflow:50000"         # /opt/airflow             — apache/airflow image declares USER airflow (50000)
    "postgres-db:70"        # /var/lib/postgresql/data — alpine postgres user (70)
)

mkdir -p "${SERVICE_DIR}/data"                                                        # the data/ parent: plain, no ACL
sudo chown "${SERVICE_USER}:${SERVICE_ADMIN}" "${SERVICE_DIR}/data"                   # owner/admin
sudo chmod 0750 "${SERVICE_DIR}/data"                                                 # children are set individually below
for entry in "${data_dirs[@]}"; do
    sub="${entry%%:*}"                                  # subdir name
    ids="${entry#*:}"                                   # "uid" or "uid:gid"
    uid="${ids%%:*}"                                    # uid
    gid="${ids#*:}"; [ "$gid" = "$ids" ] && gid="$uid"  # gid (defaults to uid)
    dir="${SERVICE_DIR}/data/${sub}"
    sudo mkdir -p "$dir"
    sudo chown -R "$((SUBUID_BASE + uid - 1)):$((SUBGID_BASE + gid - 1))" "$dir"   # map both to host subids
    sudo chmod 0700 "$dir"
done

# --- SERVICE_DIR: application data, env files, configuration -------------------
# Full treatment so the admin group gets sudo-less access AND the service user keeps
# access even on admin-created files. data/ is EXCLUDED: those dirs are owned by the
# containers' subuids and need engine-specific modes, so they were handled just above.
apply_acl_tree "${SERVICE_DIR}" "${SERVICE_DIR}/data"                                 # treat everything EXCEPT data/

Configure Secure DAG Directory Permissions

# The DAG sources are edited by the admins on the host but READ by the container
# (as uid 50000 → its mapped subuid). Like the mkdocs bare repo, the container
# gets a read-only NAMED ACL entry — never world bits.
AIRFLOW_SUBUID=$((SUBUID_BASE + 50000 - 1))

sudo mkdir -p "${SERVICE_DIR}/dags"
sudo chown "${SERVICE_USER}:${SERVICE_ADMIN}" "${SERVICE_DIR}/dags"

# container subuid: read existing items + inherit read on new items
sudo setfacl -R -m u:${AIRFLOW_SUBUID}:rX "${SERVICE_DIR}/dags"
sudo setfacl -R -d -m u:${AIRFLOW_SUBUID}:rX "${SERVICE_DIR}/dags"

Configure Secure NGINX Static Assets Permissions

# Grant the NGINX worker user (www-data) read-only access to assets subtrees.
#   $1     = service directory → traverse-only (x)
#   $2..$n = assets subtrees served directly by NGINX (configs, icons, favicon, manifest, ...)
apply_acl_nginx() {
    local tree="$1"; shift
    local assets

    sudo setfacl -m u:www-data:x "$tree"                                        # traverse only: pass through, no listing, no reading

    for assets in "$@"; do
        # assets directories — enter + list, existing and future
        sudo find "$assets" -type d -exec setfacl    -m u:www-data:rx {} +      # www-data: read existing dirs
        sudo find "$assets" -type d -exec setfacl -d -m u:www-data:rx {} +      # www-data: inherit read on new items

        # assets files — www-data read-only, owner/admin rw, never executable.
        # Entries AND mask are pinned explicitly: a bare `setfacl -m u:www-data:r`
        # would recalculate the mask to the union of all entries, resurfacing the
        # latent rwx inherited from the default ACL (files would show group rwx).
        sudo find "$assets" -type f -exec setfacl \
          -m u:www-data:r,u:${SERVICE_USER}:rw,g:${SERVICE_ADMIN}:rw,u::rw,g::rw,o::-,m::rw {} +
    done
}

# --- SERVICE_DIR/nginx + SERVICE_DIR/icons: served directly by NGINX -----------
# Created AFTER the apply_acl_tree pass so they inherit the default ACL + setgid
# group (service user/admin keep full access); www-data is then layered on top
# as a read-only named entry.
sudo mkdir -p "${SERVICE_DIR}/nginx" "${SERVICE_DIR}/icons"
sudo chown "${SERVICE_USER}:${SERVICE_ADMIN}" "${SERVICE_DIR}/nginx" "${SERVICE_DIR}/icons"
apply_acl_nginx "${SERVICE_DIR}" "${SERVICE_DIR}/nginx" "${SERVICE_DIR}/icons"

🛡️ Rootless Podman Defaults¶

Configure rootless Podman per-user configuration directory

# configure rootless Podman defaults for this service user
# - store per-user Podman configuration under ~/.config/containers
sudo -u "${SERVICE_USER}" mkdir -p "${SERVICE_HOME}/.config/containers/systemd"

Configure rootless Podman defaults

# configure rootless Podman defaults for this service user
# - use k8s-file logging for easier log rotation and inspection
sudo -u "${SERVICE_USER}" tee "${SERVICE_HOME}/.config/containers/containers.conf" >/dev/null <<EOF
[containers]
log_driver = "k8s-file"

[engine]
healthcheck_events=false
EOF

🔧 Service Configuration¶

Setup Airflow Parameters

Before deploying, you need to define a few environment variables that will be used throughout the setup process.

BASE_URL: public URL where the web service is accessible
HOST_PORT: external port used by NGINX to route traffic to the service
PG_DB / PG_USER / PG_PASSWORD: Airflow metadata database credentials

NGINX Proxy ConfigurationPostgres ConfigurationAirflow Configuration

###################################################################################
# NGINX Proxy Configuration
###################################################################################
HOST_PORT=10060

###################################################################################
# Postgres Configuration
###################################################################################
PG_DB=airflow
PG_USER=airflow
PG_PASSWORD="$(openssl rand -hex 32)"

###################################################################################
# Airflow Configuration
###################################################################################
BASE_URL=https://airflow.domain.fr

# Fernet key: encrypts Connections/Variables at rest in the metadata DB
# (must be 32 url-safe base64 bytes, hence the tr)
FERNET_KEY="$(openssl rand 32 | base64 | tr '+/' '-_')"

# JWT secret: signs the UI/API session tokens
JWT_SECRET="$(openssl rand -hex 32)"

Create Environment files

postgres.envairflow.env

sudo -u "${SERVICE_USER}" tee "${SERVICE_DIR}/postgres.env" >/dev/null <<EOF
###################################################################################
# Postgres Database Configuration
###################################################################################
POSTGRES_VERSION=16-alpine
POSTGRES_DB=${PG_DB}
POSTGRES_USER=${PG_USER}
POSTGRES_PASSWORD=${PG_PASSWORD}
EOF

sudo -u "${SERVICE_USER}" tee "${SERVICE_DIR}/airflow.env" >/dev/null <<EOF
###################################################################################
# NGINX Proxy Configuration
###################################################################################
HOST_PORT=${HOST_PORT}

###################################################################################
# Airflow Generic Configuration
###################################################################################
AIRFLOW_VERSION=latest
AIRFLOW__API__BASE_URL=${BASE_URL}
AIRFLOW__API_AUTH__JWT_SECRET=${JWT_SECRET}
AIRFLOW__CORE__FERNET_KEY=${FERNET_KEY}
AIRFLOW__CORE__DEFAULT_TIMEZONE=Europe/Paris

###################################################################################
# Airflow Executor / DAGs Configuration
###################################################################################
# LocalExecutor runs tasks as local subprocesses (no external worker);
# SequentialExecutor would only ever run a single task at a time
AIRFLOW__CORE__EXECUTOR=LocalExecutor
AIRFLOW__CORE__PARALLELISM=4
AIRFLOW__CORE__DAGS_FOLDER=/dags
AIRFLOW__CORE__LOAD_EXAMPLES=False

###################################################################################
# Airflow Idle CPU Tuning
###################################################################################
# Airflow daemons POLL (they are not event-driven like the other web services):
# the defaults wake the scheduler every 1s and re-parse the DAG files every 30s,
# which burns a constant few % of the C4's ARM cores for nothing. A
# low-traffic instance like this one does not need second-level reactivity.
# scheduler: wake every 5s instead of 1s; DB heartbeats every 60s instead of 5s.
# The wake-up drives task transitions (one grid box to the next waits up to
# one sleep), so it stays short; the heartbeats are pure liveness reporting.
AIRFLOW__SCHEDULER__SCHEDULER_IDLE_SLEEP_TIME=5
AIRFLOW__SCHEDULER__SCHEDULER_HEARTBEAT_SEC=60
AIRFLOW__SCHEDULER__JOB_HEARTBEAT_SEC=60
AIRFLOW__API__WORKERS=1
# keep the health endpoint truthful with the slower heartbeat (default 30s
# would report a scheduler that beats every 60s as dead)
AIRFLOW__SCHEDULER__SCHEDULER_HEALTH_CHECK_THRESHOLD=150
# dag-processor: re-parse the DAG files every 10min instead of 30s — an edit
# of a DAG file now takes up to 10min to appear in the UI
AIRFLOW__DAG_PROCESSOR__MIN_FILE_PROCESS_INTERVAL=600
AIRFLOW__DAG_PROCESSOR__REFRESH_INTERVAL=600
AIRFLOW__DAG_PROCESSOR__PARSING_PROCESSES=1
# the main loop is only one of several timers: each of these runs its own
# Python + SQLAlchemy + Postgres round-trip on its own clock — push the
# frequent ones out (pool metrics 5s, zombie scan 10s, trigger check 15s,
# parsing cleanup 60s, dag-processor stats log 30s)
AIRFLOW__SCHEDULER__POOL_METRICS_INTERVAL=300
AIRFLOW__SCHEDULER__TASK_INSTANCE_HEARTBEAT_TIMEOUT_DETECTION_INTERVAL=300
AIRFLOW__SCHEDULER__TRIGGER_TIMEOUT_CHECK_INTERVAL=300
AIRFLOW__SCHEDULER__PARSING_CLEANUP_INTERVAL=300
AIRFLOW__DAG_PROCESSOR__PRINT_STATS_INTERVAL=600
# less log churn (every INFO line costs journald/k8s-file writes).
# Side effect: the task logs of SUCCESSFUL tasks lose their INFO lines;
# errors land on stderr and stay visible, as does any failure.
# Set back to INFO for full task logs.
AIRFLOW__LOGGING__LOGGING_LEVEL=WARNING

###################################################################################
# Airflow Metadata Database Configuration
###################################################################################
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN=postgresql+psycopg2://${PG_USER}:${PG_PASSWORD}@airflow-postgres:5432/${PG_DB}
EOF

Keep the .env files

All the secret informations will be stored in the .env files.

⚙️ Create Airflow build¶

Create Airflow build Dockerfile

build/Dockerfile

# setup of Dockerfile
mkdir -p "${SERVICE_DIR}/build"
tee "${SERVICE_DIR}/build/Dockerfile" > /dev/null <<'EOF'
#############################################
##### Runtime image
#############################################
ARG AIRFLOW_VERSION=latest
FROM docker.io/apache/airflow:${AIRFLOW_VERSION}

#############################################
##### Install any extra Airflow providers or
##### Python packages your DAGs need here, e.g.
#############################################
# RUN pip install --no-cache-dir "apache-airflow-providers-ssh"
EOF

🧩 Quadlet Service¶

Create Network Podman Quadlet

airflow.network

# podman quadlet: create network
sudo -u "${SERVICE_USER}" tee "${SERVICE_HOME}/.config/containers/systemd/airflow.network" >/dev/null <<EOF
[Network]
NetworkName=airflow
EOF

Create Postgres Podman Quadlet

airflow-postgres.container

# podman quadlet: create PostgreSQL
sudo -u "${SERVICE_USER}" tee "${SERVICE_HOME}/.config/containers/systemd/airflow-postgres.container" >/dev/null <<'EOF'
[Unit]
Description=airflow PostgreSQL database

[Container]
EnvironmentFile=/media/ssd/podman/airflow/postgres.env
Image=docker.io/library/postgres:${POSTGRES_VERSION}
ContainerName=airflow-postgres
Network=airflow.network

# run as the postgres uid/gid (alpine = 70). The image starts its entrypoint as root
# then drops to 70 before creating PGDATA, so we pin the uid here. The data dir is
# pre-chowned to 70's mapped subuid in the permissions step, so a plain :rw mount works.
User=70:70

Environment=PGDATA=/var/lib/postgresql/data/pgdata

# plain bind mount (data dir already owned by 70's subuid). Host data is owned by that
# subuid; the debian admin still reaches it via the SERVICE_DIR ACLs / sudo.
Volume=/media/ssd/podman/airflow/data/postgres-db:/var/lib/postgresql/data:rw

HealthCmd=pg_isready -U $POSTGRES_USER -d $POSTGRES_DB
HealthStartPeriod=30s
HealthInterval=60s
HealthTimeout=5s
HealthRetries=3
HealthOnFailure=kill

[Service]
EnvironmentFile=/media/ssd/podman/airflow/postgres.env
UMask=0007

Restart=always
RestartSec=30
TimeoutStartSec=180

[Install]
WantedBy=default.target
EOF

UMask=0007 should be configured for each Podman Quadlet service. It removes all permissions for others while preserving read/write access for the service owner and admin group on newly created files and directories.

Create Build Podman Quadlet

airflow.build

# podman quadlet: create airflow build
sudo -u "${SERVICE_USER}" tee "${SERVICE_HOME}/.config/containers/systemd/airflow.build" >/dev/null <<'EOF'
[Unit]
Description=Build airflow image

[Build]
ImageTag=localhost/airflow:${AIRFLOW_VERSION}
File=/media/ssd/podman/airflow/build/Dockerfile
SetWorkingDirectory=/media/ssd/podman/airflow/build
PodmanArgs=--build-arg AIRFLOW_VERSION=${AIRFLOW_VERSION}

[Service]
EnvironmentFile=/media/ssd/podman/airflow/airflow.env
# After a successful (re)build, drop now-untagged old image versions.
# Dangling-only: the tagged airflow:${AIRFLOW_VERSION} and its layers are kept.
ExecStartPost=-/usr/bin/podman image prune -f
EOF

One container per Airflow component — and no triggerer

airflow standalone is not used: it always launches the triggerer, an always-on asyncio loop that burns a constant few % of CPU and is only needed by deferrable operators, which this setup does not use. Each component therefore gets its own Quadlet, and the triggerer simply does not exist:

Container	Role
`airflow-scheduler`	migrates the metadata schema (idempotent), then schedules the task runs
`airflow-dag-processor`	parses `/dags` and serializes the DAGs to the database
`airflow-server`	the api-server: web UI + REST API (generates the admin password)

Create Scheduler Podman Quadlet

airflow-scheduler.container

# podman quadlet: create airflow scheduler
sudo -u "${SERVICE_USER}" tee "${SERVICE_HOME}/.config/containers/systemd/airflow-scheduler.container" >/dev/null <<'EOF'
[Unit]
Description=airflow scheduler
Requires=airflow-build.service airflow-postgres.service
After=airflow-build.service airflow-postgres.service

[Container]
EnvironmentFile=/media/ssd/podman/airflow/airflow.env
Image=airflow.build
ContainerName=airflow-scheduler
Network=airflow.network

# No User= needed: the apache/airflow image already declares USER airflow (50000).
# The data dir is pre-chowned to airflow's mapped subuid in the permissions step.

# migrate the metadata schema (idempotent, fast when up to date), then schedule
Exec=bash -c "airflow db migrate && exec airflow scheduler"

# plain bind mount
#   AIRFLOW_HOME: owned by 50000's subuid
#   DAG sources: admin-edited on the host, read-only named-ACL access for the container
Volume=/media/ssd/podman/airflow/data/airflow:/opt/airflow:rw
Volume=/media/ssd/podman/airflow/dags:/dags:ro

[Service]
EnvironmentFile=/media/ssd/podman/airflow/airflow.env
UMask=0007
SuccessExitStatus=143

Restart=always
RestartSec=30
TimeoutStartSec=900
# wait until Postgres reports healthy before migrating/scheduling
ExecStartPre=/bin/sh -c 'until podman healthcheck run airflow-postgres >/dev/null 2>&1; do sleep 5; done'

[Install]
WantedBy=default.target
EOF

No HealthCmd on purpose: the only probe Airflow offers here is the airflow jobs check CLI, which spawns a full Python interpreter at every interval — a permanent CPU cost on the ARM board that would defeat the idle tuning. Restart=always already resurrects a crashed scheduler.

Create DAG-Processor Podman Quadlet

airflow-dag-processor.container

# podman quadlet: create airflow dag-processor
sudo -u "${SERVICE_USER}" tee "${SERVICE_HOME}/.config/containers/systemd/airflow-dag-processor.container" >/dev/null <<'EOF'
[Unit]
Description=airflow dag-processor
Requires=airflow-scheduler.service
After=airflow-scheduler.service

[Container]
EnvironmentFile=/media/ssd/podman/airflow/airflow.env
Image=airflow.build
ContainerName=airflow-dag-processor
Network=airflow.network

# No User= needed: the apache/airflow image already declares USER airflow (50000).

Exec=dag-processor

# plain bind mount
#   AIRFLOW_HOME: owned by 50000's subuid
#   DAG sources: admin-edited on the host, read-only named-ACL access for the container
Volume=/media/ssd/podman/airflow/data/airflow:/opt/airflow:rw
Volume=/media/ssd/podman/airflow/dags:/dags:ro

[Service]
EnvironmentFile=/media/ssd/podman/airflow/airflow.env
UMask=0007
SuccessExitStatus=143

Restart=always
RestartSec=30
TimeoutStartSec=900

[Install]
WantedBy=default.target
EOF

Create Airflow Server Podman Quadlet

airflow-server.container

# podman quadlet: create airflow api-server
sudo -u "${SERVICE_USER}" tee "${SERVICE_HOME}/.config/containers/systemd/airflow-server.container" >/dev/null <<'EOF'
[Unit]
Description=airflow api-server
Requires=airflow-scheduler.service airflow-dag-processor.service
After=airflow-scheduler.service airflow-dag-processor.service

[Container]
EnvironmentFile=/media/ssd/podman/airflow/airflow.env
Image=airflow.build
ContainerName=airflow-server
Network=airflow.network

# No User= needed: the apache/airflow image already declares USER airflow (50000).

Exec=api-server

# plain bind mount
#   AIRFLOW_HOME: owned by 50000's subuid (logs are read from here)
Volume=/media/ssd/podman/airflow/data/airflow:/opt/airflow:rw

PublishPort=127.0.0.1:${HOST_PORT}:8080

HealthCmd=curl -fsS -o /dev/null http://localhost:8080/api/v2/monitor/health
HealthStartPeriod=300s
HealthInterval=60s
HealthTimeout=5s
HealthRetries=3
HealthOnFailure=kill

[Service]
EnvironmentFile=/media/ssd/podman/airflow/airflow.env
UMask=0007
SuccessExitStatus=143

Restart=always
RestartSec=30
TimeoutStartSec=900

[Install]
WantedBy=default.target
EOF

UMask=0007: removes all permissions for others while preserving read/write access for the service owner and admin group on newly created files and directories SuccessExitStatus=143: treat a SIGTERM-style exit (143) as a clean stop, not a failure First start only: the schema migration runs inside airflow-scheduler; if the dag-processor or the api-server come up before it finishes, they exit and are restarted 30s later by systemd until the schema is ready — a few restart cycles on the very first boot are expected.

▶️ Start Airflow¶

Enable and Start Quadlet Services

# open an interactive shell as the service user
sudo -iu "${SERVICE_USER}"

# reload systemd user units
systemctl --user daemon-reload

# start Podman Quadlet services (Requires= pulls postgres, scheduler
# and dag-processor automatically)
systemctl --user start airflow-server.service

# verify service status
systemctl --user status airflow-server.service airflow-scheduler.service airflow-dag-processor.service

To follow the service logs in real time, run sudo journalctl _UID=$(id -u ${SERVICE_USER}) -f from the Debian account.

Retrieve the generated admin password

# the api-server (SimpleAuthManager) generates the admin password on first start
# (the file persists in the AIRFLOW_HOME bind mount)
sudo cat /media/ssd/podman/airflow/data/airflow/simple_auth_manager_passwords.json.generated

Login with user admin and the generated password.

🚀 Deploy Airflow¶

Install NGINX

NGINX needs to be installed, follow the NGINX section.

Configure NGINX

NGINX needs to be configured using a file in /etc/nginx/sites-enabled directory.
This configuration file specify the documentation path:

/etc/nginx/sites-enabled/airflow.domain.frActivate website

server {
  server_name airflow.domain.fr;

  # setup 404 error_page
  error_page 404 /404.html;
  include snippets/error-404.conf;

  # show maintenance page when backend is down
  error_page 502 503 504 /maintenance.html;
  include snippets/error-maintenance.conf;

  # reverse proxy
  location / {
    proxy_pass http://127.0.0.1:10060;

    # keep it HTTP/1.1
    proxy_http_version 1.1;

    # websocket support (live UI updates and log streaming)
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";

    # forwarded headers
    include proxy_params;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Forwarded-Port $server_port;

    # application-specific tuning
    proxy_read_timeout 3600s;
    proxy_send_timeout 3600s;
    proxy_buffering off;
    proxy_redirect off;
  }
}

# restart nginx
sudo nginx -t && sudo service nginx restart

Replace airflow.domain.fr by the name of your website.

Activate HTTPS

To activate HTTPS protocol, follow the Let's Encrypt section.