💾 Lightflow Backup¶

Lightflow is a generic task runner: the backup logic lives entirely in the scripts it runs and the host-side gate they reach over SSH. This tutorial configures an installed Lightflow instance to drive the nightly OneDrive backups of the Podman services (calibre, immich, kouizine, planka, vaultwarden) and of Lightflow itself — replacing the heavier Airflow stack while keeping the exact same security model: Lightflow never reads another service's files and never holds the OneDrive token, the boundary lives in backupctl on the host.

📋 Requirements¶

Info

Lightflow Backup requires the installation of:

Lightflow
podmanctl (used by backupctl to stop/start the service stacks)

🛤️ End-to-end path¶

How a nightly run travels from the scheduler all the way to OneDrive — and who enforces what at each hop:

Only backupctl, as root on the host, touches service files and the OneDrive token; Lightflow's own task dumps its database first and ships data/backup alone.

🗂️ The four-step backup model¶

For each backed-up service the nightly run executes four steps, in order:

Step	Action	Box
`stop`	`podmanctl --stop <svc>` (whole stack → consistent files)	🟩 / 🟥
`sync`	`rclone sync data/ → OneDrive` (mirror)	🟩 / 🟥
`archive`	`tar\\|gzip\\|rclone rcat` retention, Sundays only	🟩 / 🟥 / ⬜ (grey other days)
`start`	`podmanctl --start <svc>` run rule `always`	🟩 / 🟥

A step's exit code drives its box: 0 → success 🟩, 75 → skipped ⬜, anything else → failed 🟥. The start step uses the always run rule, so the service is restarted even if a previous step failed.

🪶 Lightflow's own backup¶

Lightflow backs itself up with three steps and no stop/start: it cannot stop the very container running the task, and needs not to — pg_dump takes a transactionally consistent snapshot of the live database.

Step	Action	Box
`dump`	`backup.py --dump`, inside the container `pg_dump` → `/data/backup/lightflow.dump` + `/scripts` → `/data/backup/scripts.tar.gz`	🟩 / 🟥
`sync`	`backup.py --sync lightflow data/backup` (mirror of `data/backup` only)	🟩 / 🟥
`archive`	`backup.py --archive lightflow data/backup` retention, Sundays only	🟩 / 🟥 / ⬜

Only data/backup leaves the host. data/ as a whole holds the live postgres-db cluster: tar fails on it (file changed as we read it) and a raw copy of a running cluster could not be restored anyway. python-libs/ is reproducible from the Python packages page.
No Variable, no password. The image ships the PostgreSQL 18 client and every step inherits the server's DATABASE_URL; /data/backup is the data/backup volume, so backupctl reads the dump on the host side.
Both files are written under a temporary name and renamed on success: a failed dump never replaces the last good copy.

🔐 Backup Gate¶

The host-side plumbing that the DAGs call through SSH:

rclone (OneDrive transfers)
backupctl (the validating gate)
backup_ops SSH account.

☁️ OneDrive Remote¶

Install rclone and configure the OneDrive remote

Install rcloneConfigure the remote (interactive)Move the config to /etc/backupctl

# install rclone (OneDrive transfers are driven by backupctl)
sudo apt update
sudo apt install -y rclone

# the ODROID is headless: the Microsoft OAuth consent needs a graphical
# browser, and its redirect is hard-wired to localhost:53682 ON THE
# MACHINE RUNNING RCLONE. An SSH tunnel makes the desktop browser's
# localhost reach the ODROID, so everything stays on the device.
#
# 1. on the DESKTOP (Windows 11: OpenSSH is built in), open the tunnel
#    and KEEP THE SESSION OPEN:
#      ssh -L 53682:localhost:53682 debian@odroid
#
# 2. inside that SSH session, run the interactive configuration:
rclone config
#      n) New remote
#      name>             onedrive
#      Storage>          onedrive
#      client_id>        (empty)
#      client_secret>    (empty)
#      region>           global
#      Edit advanced config?            n
#      Use web browser to authenticate? y   ← yes: the tunnel carries the redirect
#
# 3. rclone cannot open a browser on the ODROID: it prints the URL
#      http://127.0.0.1:53682/auth?state=...
#    open it in the DESKTOP browser and sign in to the Microsoft
#    account — the localhost redirect flows back through the tunnel
#    and rclone resumes automatically:
#
#      Your type of connection>         onedrive (personal or business)
#      Drive to use>     (select your drive → fills drive_id / drive_type)
#      Keep this "onedrive" remote?     y

# the live token belongs to root only: backupctl is the single consumer
sudo mkdir -p /etc/backupctl
sudo mv ~/.config/rclone/rclone.conf /etc/backupctl/rclone.conf
sudo chown root:root /etc/backupctl/rclone.conf
sudo chmod 0600 /etc/backupctl/rclone.conf

# verify the remote and create the backup folder
sudo rclone --config /etc/backupctl/rclone.conf lsd onedrive:
sudo rclone --config /etc/backupctl/rclone.conf mkdir onedrive:/OdroidBackup

The live OAuth token rotates — never re-apply an old copy

Microsoft rotates the refresh token on every use: rclone persists each rotation by rewriting /etc/backupctl/rclone.conf. That file (root, 0600) is the single source of truth for the token, the LightFlow backup_onedrive Variable never holds it (it only carries remote/folder). Keep a copy of the working config in Bitwarden.

🎛️ Install backupctl¶

Install backupctl to /usr/local/bin

backupctl

# download the script published by this documentation site
sudo curl -fsSL https://docs.fum-server.fr/files/backupctl.py -o /usr/local/bin/backupctl

# make it executable for everyone, writable by root only
sudo chmod 0755 /usr/local/bin/backupctl

# check the installation (backupctl refuses to run without root)
sudo backupctl --help

The script is plain Python 3 standard library.

👤 Create the backup_ops SSH account¶

Create the backup_ops user and sudoers rule

# dedicated account used ONLY by the LightFlow container to reach backupctl.
# NOT added to the no-ssh group on purpose: SSH is its entire reason to exist,
# and the forced command below is the only thing it can ever execute.
sudo useradd --system --create-home --home-dir /home/backup_ops --shell /bin/bash backup_ops

# sudoers drop-in: keep the client's original command line visible to the
# gate (sshd exports it as SSH_ORIGINAL_COMMAND) and allow exactly ONE
# invocation as root — no wildcard
sudo tee /etc/sudoers.d/backupctl >/dev/null <<'EOF'
Defaults!/usr/local/bin/backupctl env_keep += "SSH_ORIGINAL_COMMAND"
backup_ops ALL=(root) NOPASSWD: /usr/local/bin/backupctl --ssh-gate
EOF
sudo chmod 0440 /etc/sudoers.d/backupctl

# validate the sudoers syntax before it can lock you out
sudo visudo -cf /etc/sudoers.d/backupctl

🔑 Authorize Lightflow's SSH key¶

Lightflow doesn't keep a private key file on disk: the key material lives in the BACKUP_SSH_KEY Variable (secret) and backup.py writes it to a private, short-lived temp file only while ssh runs. The same applies to the host-key line in BACKUP_KNOWN_HOSTS. This keeps the script generic, a future task that reaches another device just gets its own key Variable, with nothing to mount.

Generate a key pair, authorize it, and capture the values to paste

# generate a throwaway key pair in a temp dir, the PRIVATE key will live only
# in the BACKUP_SSH_KEY Variable, never as a file on the host or a mount
TMP="$(mktemp -d)"
ssh-keygen -t ed25519 -N "" -C "lightflow-backup" -f "${TMP}/id_ed25519"

# authorize the PUBLIC key for backup_ops with the SAME forced command
PUBKEY="$(cat ${TMP}/id_ed25519.pub)"
sudo tee -a /home/backup_ops/.ssh/authorized_keys >/dev/null <<EOF
restrict,command="sudo /usr/local/bin/backupctl --ssh-gate" ${PUBKEY}
EOF
sudo chmod 0600 /home/backup_ops/.ssh/authorized_keys
sudo chown backup_ops:backup_ops /home/backup_ops/.ssh/authorized_keys

# ---- value for BACKUP_KNOWN_HOSTS -------------------------------------------
# the container reaches the host as host.containers.internal, so the known_hosts
# line must carry THAT name in front of the host's own ed25519 public host key
printf 'host.containers.internal %s\n' \
  "$(sudo awk '{print $1, $2}' /etc/ssh/ssh_host_ed25519_key.pub)"

# ---- value for BACKUP_SSH_KEY -----------------------------------------------
# print the PRIVATE key to copy, then wipe the temp dir so nothing lingers
cat "${TMP}/id_ed25519"
rm -rf "${TMP}"

Paste the private key into the BACKUP_SSH_KEY Variable (secret) and the host.containers.internal … line into BACKUP_KNOWN_HOSTS. No private key file is kept on the host and no .ssh volume is mounted. backupctl accepts only --stop/--start/--sync/--archive over the gate. --restore stays a manual, on-host operation.

⚙️ Configure Lightflow (in the UI)¶

1. Create the backup pool

Pools → Add pool

Name: backup
Slots: 1

A 1-slot pool serializes every backup step across all services Two steps can never run at once.

2. Create the backup variables

Variables → Add variable (these become environment variables for the scripts):

Key	Value	Secret
`BACKUP_HOST`	`backup_ops@host.containers.internal`	no
`BACKUP_SSH_KEY`	(paste the SSH private key* contents — see below)*	yes
`BACKUP_KNOWN_HOSTS`	(paste the `known_hosts` host-key line — see below)	no
`ONEDRIVE_REMOTE`	`onedrive`	no
`ONEDRIVE_FOLDER`	`/OdroidBackup`	no
`ARCHIVE_WEEKDAY`	`sunday`	no
`KEEP_WEEKLY`	`3`	no
`KEEP_MONTHLY`	`12`	no

The OneDrive token is not stored here. It rotates in /etc/backupctl/rclone.conf on the host. Nor is the database URL: --dump uses the server's own DATABASE_URL, which every step inherits.

3. Create the backup scripts

backup.py

#!/usr/bin/env python3
"""
backup.py — sample Lightflow task script (OneDrive backup of a Podman service).

Lightflow itself is a generic runner: this script holds the backup logic and
reaches the host EXACTLY like the old Airflow DAGs did — over the restricted
`backup_ops` SSH forced command, which runs `sudo backupctl --ssh-gate` on the
host. Lightflow never reads another service's files and never holds the OneDrive
token; the security boundary lives in `backupctl` on the host.

One Lightflow TASK per service, made of four ordered STEPS — each step runs this
script with one action and shows up as one box in the grid:

    Step 1  stop      python3 /scripts/backup.py --stop    <service>     (on_success)
    Step 2  sync      python3 /scripts/backup.py --sync    <service>     (on_success)
    Step 3  archive   python3 /scripts/backup.py --archive <service>     (on_success)
    Step 4  start     python3 /scripts/backup.py --start   <service>     (always)

`archive` exits 75 (Lightflow's "skipped"/grey box) on non-archive days, so the
weekly archive shows grey except on ARCHIVE_WEEKDAY. `start` uses the `always`
run rule so the service is restarted even if a previous step failed.

`--sync` and `--archive` ship the service's `data` directory by default; an
optional third argument names another directory below the service root instead.

Lightflow backs ITSELF up with three steps and no stop/start — it cannot stop
the very container running the task, and needs not to: `--dump` takes a hot,
transactionally consistent pg_dump of its own database (plus a snapshot of the
scripts) into /data/backup, and only that directory is shipped. The live
data/postgres-db is never shipped: a raw copy of a running cluster is unusable.

    Step 1  dump      python3 /scripts/backup.py --dump                             (on_success)
    Step 2  sync      python3 /scripts/backup.py --sync    lightflow data/backup    (on_success)
    Step 3  archive   python3 /scripts/backup.py --archive lightflow data/backup    (on_success)

`--dump` runs locally, inside the Lightflow container: the image ships the
PostgreSQL 18 client and every step inherits the server's DATABASE_URL, so it
needs no SSH and no Variable.

A non-service directory on the host (e.g. a git tree owned by www-data:debian)
is mirrored with a single step — it has no stack to stop/start:

    sync-extra   python3 /scripts/backup.py --sync-extra <target>   (on_success)

`<target>` is a NAME from backupctl's EXTRA_TARGETS allow-list (the host resolves
its path; we never pass --path); the mirror lands under <remote>/extra/<target>/.

Configuration comes from Lightflow VARIABLES (injected as environment variables):

    BACKUP_HOST        backup_ops@host.containers.internal   (required)
    BACKUP_SSH_KEY     SSH private key — inline PEM value OR a file path
    BACKUP_KNOWN_HOSTS known_hosts entry — inline value OR a file path
    ONEDRIVE_REMOTE    onedrive                              (default)
    ONEDRIVE_FOLDER    /OdroidBackup                         (default)
    ARCHIVE_WEEKDAY    sunday                                (default)
    KEEP_WEEKLY        3                                     (default)
    KEEP_MONTHLY       12                                    (default)

The timeout knob is set ONCE per step, in the UI: Lightflow enforces it in
the container (SIGTERM the step's process group, SIGKILL 10s later) and
injects it as LIGHTFLOW_STEP_TIMEOUT. This script derives the host-side
`backupctl --timeout` from it — slightly smaller, so backupctl aborts
cleanly (killing its own children) before Lightflow tears the group down,
and the host operation can never outlive a dropped SSH session.

Holding the SSH key (and known_hosts) inline in a secret Variable keeps the
script generic — any future device just needs its own key Variable, with no
host-side key files to provision. The key material is written to a private,
short-lived temp file only while `ssh` runs, then removed.
"""

import os
import re
import subprocess
import sys
import tarfile
import tempfile
import textwrap
from datetime import date
from pathlib import Path, PurePosixPath
from urllib.parse import unquote, urlsplit

SKIP_EXIT_CODE = 75  # Lightflow renders this as a grey "skipped" box
ACTIONS = {"--stop", "--start", "--sync", "--archive", "--sync-extra", "--dump"}

# --dump: the backup volume (data/backup on the host) and what lands in it
BACKUP_DIR = Path("/data/backup")
DUMP = BACKUP_DIR / "lightflow.dump"
SCRIPTS_ARCHIVE = BACKUP_DIR / "scripts.tar.gz"


def env(key, default=None):
    return os.environ.get(key, default)


def materialize(value, kind):
    """Resolve a key / known_hosts VALUE to a file path that `ssh` can use.

    A Variable may hold either a filesystem PATH (legacy) or the INLINE value
    itself (preferred — generic, no host-side key files). Inline content is
    written to a private (0600) temp file; the caller removes it afterwards.
    A private key is recognised by its PEM header; a known_hosts entry by the
    whitespace separating its fields. A key whose line breaks were flattened
    to spaces by a single-line input field is re-wrapped into a valid PEM.
    Returns (path, is_temp).
    """
    # normalize line endings: a value pasted or stored with Windows CRLF
    # (or a stray \r) keeps \r inside the body, so force LF.
    text = value.replace("\r\n", "\n").replace("\r", "\n").strip()
    inline = "-----BEGIN" in text if kind == "key" else any(c.isspace() for c in text)
    if not inline:
        return value, False
    if kind == "key":
        # Some Variable fields flatten a multi-line PEM onto ONE line, turning
        # the newlines into spaces — ssh then fails with "error in libcrypto".
        # Rebuild a canonical PEM: keep the BEGIN/END markers, strip ALL
        # whitespace from the base64 body, then re-wrap at 70 columns. This is
        # a no-op on an already-correct key, so it is always safe to run.
        m = re.search(r"-----BEGIN ([A-Z0-9 ]+)-----(.*)-----END \1-----", text, re.S)
        if m:
            label = m.group(1)
            body = re.sub(r"\s+", "", m.group(2))
            text = "-----BEGIN {0}-----\n{1}\n-----END {0}-----".format(
                label, "\n".join(textwrap.wrap(body, 70))
            )
    fd, path = tempfile.mkstemp(prefix="lightflow-", suffix=f".{kind}")
    with os.fdopen(fd, "w") as f:
        f.write(text + "\n")
    os.chmod(path, 0o600)
    return path, True


def host_timeout():
    """Host-side backupctl deadline derived from the ONE timeout knob: the
    step timeout Lightflow injects as LIGHTFLOW_STEP_TIMEOUT. Slightly
    below it, so backupctl aborts cleanly (killing its own children)
    before Lightflow tears down the process group; the 30s margin is
    capped at a quarter of the step timeout so short steps keep most of
    their budget."""
    step = int(env("LIGHTFLOW_STEP_TIMEOUT", "3600"))
    return max(10, step - min(30, step // 4))


def ssh_backupctl(args):
    """Run `backupctl <args> --timeout <derived>` on the host via the SSH forced command."""
    host = env("BACKUP_HOST")
    if not host:
        sys.exit("error: BACKUP_HOST is not set (e.g. backup_ops@host.containers.internal)")
    key, key_tmp = materialize(env("BACKUP_SSH_KEY", "/data/.ssh/id_ed25519"), "key")
    known, known_tmp = materialize(env("BACKUP_KNOWN_HOSTS", "/data/.ssh/known_hosts"), "known_hosts")

    remote = "backupctl " + " ".join(args) + f" --timeout {host_timeout()}"
    cmd = [
        "ssh",
        "-i", key,
        "-o", "BatchMode=yes",
        "-o", "StrictHostKeyChecking=accept-new",
        "-o", f"UserKnownHostsFile={known}",
        host,
        remote,
    ]
    print("+ " + " ".join(cmd), flush=True)
    try:
        return subprocess.run(cmd).returncode
    finally:
        for path, is_tmp in ((key, key_tmp), (known, known_tmp)):
            if is_tmp:
                try:
                    os.unlink(path)
                except OSError:
                    pass


def libpq_env(url):
    """Hand the connection to pg_dump through libpq's PG* variables, so the
    password never appears on a command line."""
    parts = urlsplit(url)
    pg = dict(os.environ)
    pg.update({
        "PGHOST": parts.hostname or "",
        "PGPORT": str(parts.port or 5432),
        "PGUSER": unquote(parts.username or ""),
        "PGPASSWORD": unquote(parts.password or ""),
        "PGDATABASE": unquote(parts.path.lstrip("/")),
    })
    return pg


def partial(path):
    return path.with_name(path.name + ".partial")


def dump():
    """Hot self-backup of Lightflow, run INSIDE its own container: a consistent
    pg_dump of the database plus a snapshot of the scripts, both written under
    a temporary name and renamed only on success — a failed run never replaces
    the last good copy. Returns the step's exit code."""
    url = env("DATABASE_URL")
    if not url:
        print("error: DATABASE_URL is not set (it comes from the server's environment)", file=sys.stderr)
        return 1
    BACKUP_DIR.mkdir(parents=True, exist_ok=True)

    # custom format: compressed, and pg_restore can select tables from it
    tmp = partial(DUMP)
    result = subprocess.run(
        ["pg_dump", "--format=custom", "--no-owner", f"--file={tmp}"],
        env=libpq_env(url),
    )
    if result.returncode != 0:
        tmp.unlink(missing_ok=True)
        print(f"error: pg_dump exited with code {result.returncode}", file=sys.stderr)
        return 1
    tmp.replace(DUMP)
    print(f"dump written: {DUMP} ({DUMP.stat().st_size} bytes)", flush=True)

    # the scripts are ordinary files on their own volume: archive them under
    # `scripts/` so a restore extracts them straight back into /scripts
    scripts = Path(env("LIGHTFLOW_SCRIPTS_DIR", "/scripts"))
    tmp = partial(SCRIPTS_ARCHIVE)
    try:
        with tarfile.open(tmp, "w:gz") as tar:
            tar.add(scripts, arcname="scripts",
                    filter=lambda info: None if "__pycache__" in info.name.split("/") else info)
    except OSError as e:
        tmp.unlink(missing_ok=True)
        print(f"error: cannot archive {scripts}: {e}", file=sys.stderr)
        return 1
    tmp.replace(SCRIPTS_ARCHIVE)
    print(f"scripts archived: {SCRIPTS_ARCHIVE} ({SCRIPTS_ARCHIVE.stat().st_size} bytes)", flush=True)
    return 0


def service_path(service, subdir):
    """The host directory to ship: <service>/data by default, or another
    directory below the service root (e.g. data/backup for Lightflow itself).
    backupctl enforces the real confinement; this only fails early and clearly."""
    sub = PurePosixPath(subdir)
    if sub.is_absolute() or ".." in sub.parts or not sub.parts:
        sys.exit(f"error: sub-directory {subdir!r} must be relative to the service root, without '..'")
    return f"/media/ssd/podman/{service}/{sub}"


def main():
    if len(sys.argv) == 2 and sys.argv[1] == "--dump":
        sys.exit(dump())
    if len(sys.argv) not in (3, 4) or sys.argv[1] not in ACTIONS - {"--dump"}:
        sys.exit("usage: backup.py --dump\n"
                 "       backup.py <--stop|--start|--sync|--archive|--sync-extra> <service|target> [subdir]")
    action, service = sys.argv[1], sys.argv[2]
    if not service.strip():
        sys.exit("error: service name is empty")
    if len(sys.argv) == 4 and action not in ("--sync", "--archive"):
        sys.exit(f"error: {action} takes no sub-directory (only --sync and --archive do)")

    path = service_path(service, sys.argv[3] if len(sys.argv) == 4 else "data")
    remote = f"{env('ONEDRIVE_REMOTE', 'onedrive')}:{env('ONEDRIVE_FOLDER', '/OdroidBackup')}"

    if action == "--stop":
        rc = ssh_backupctl(["--stop", service])
    elif action == "--start":
        rc = ssh_backupctl(["--start", service])
    elif action == "--sync":
        rc = ssh_backupctl(["--sync", service, "--path", path, "--remote", remote])
    elif action == "--sync-extra":
        # `service` is the extra-target NAME here; the host resolves its path
        # from backupctl's EXTRA_TARGETS allow-list, so we never pass --path.
        rc = ssh_backupctl(["--sync-extra", service, "--remote", remote])
    elif action == "--archive":
        weekday = env("ARCHIVE_WEEKDAY", "sunday").lower()
        if date.today().strftime("%A").lower() != weekday:
            print(f"skip: weekly archive only runs on {weekday}", flush=True)
            sys.exit(SKIP_EXIT_CODE)
        rc = ssh_backupctl([
            "--archive", service, "--path", path, "--remote", remote,
            "--keep-weekly", env("KEEP_WEEKLY", "3"),
            "--keep-monthly", env("KEEP_MONTHLY", "12"),
        ])
    else:
        sys.exit(f"error: unknown action {action!r} (stop|start|sync|archive|sync-extra|dump)")

    sys.exit(rc)


if __name__ == "__main__":
    main()

4. Create one task per service

Tasks → New task — example for vaultwarden:

Name: backup_vaultwarden
CRON: 0 3 * * * (daily 03:00) — or stagger services a few minutes apart
Pool: backup
Steps (each runs python3 backup.py <action> vaultwarden):

#	Name	Command	Script	Arguments	Run rule	Timeout
1	stop	`python3`	`backup.py`	`--stop vaultwarden`	on_success	30
2	sync	`python3`	`backup.py`	`--sync vaultwarden`	on_success	300
3	archive	`python3`	`backup.py`	`--archive vaultwarden`	on_success	600
4	start	`python3`	`backup.py`	`--start vaultwarden`	always	180

archive exits 75 (grey ⬜ "skipped") on non-ARCHIVE_WEEKDAY days. start uses always so the stack is restarted even if an earlier step failed. The Timeout column is the single timeout knob: Lightflow enforces it in the container (SIGTERM the process group → 10s → SIGKILL) and backup.py derives a slightly smaller host-side backupctl --timeout from it automatically. Duplicate the task for each service (calibre, immich, planka, …), changing the name and the service argument. The 1-slot backup pool keeps them serialized.

5. Create the Lightflow self-backup task

Tasks → New task:

Name: backup_lightflow
CRON: 0 2 * * * (daily 02:00) — any slot works, nothing is stopped
Pool: backup
Steps:

#	Name	Command	Script	Arguments	Run rule	Timeout
1	dump	`python3`	`backup.py`	`--dump`	on_success	300
2	sync	`python3`	`backup.py`	`--sync lightflow data/backup`	on_success	300
3	archive	`python3`	`backup.py`	`--archive lightflow data/backup`	on_success	600

No stop/start: the dump is hot and consistent, and stopping lightflow would kill the very container running the task. The third argument ships data/backup instead of data/ — never the live postgres-db. backupctl still confines it strictly below /media/ssd/podman/lightflow/.

💾 Operations¶

Run and monitor the backups

scheduled: each backup_<service> task fires on its CRON; the Auto toggle enables/disables scheduling without deleting the task
on demand: open a task → ▶ Run now (or ▶ Run on the Tasks page)
grid: 🟩 step ok — 🟥 step failed/timed out (the service is still restarted by start) — ⬜ archive skipped (not the archive weekday)
logs: open a task → pick a run column → pick a step → the captured backupctl output (rclone stats included) streams live while it runs
OneDrive layout: /OdroidBackup/<service>/data/ (mirror) and /OdroidBackup/<service>/archives/<service>-YYYY-MM-DD.tar.gz — for lightflow, the mirror holds lightflow.dump + scripts.tar.gz and each archive a backup/ folder with both; extra targets land under /OdroidBackup/extra/<name>/data/

First run — smoke test

Trigger the smallest service first (e.g. backup_vaultwarden) and check:

the four boxes go green and the service is back up (podmanctl --list)
the mirror appeared on OneDrive under /OdroidBackup/vaultwarden/data/
trigger it again on a non-archive day: archive must show skipped (grey)

Restore a backup (manual, on the host)

Restoring stays a deliberate on-host operation through backupctl --restore and is refused over the SSH gate on purpose — run it directly on the ODROID from the Debian account.

Restore Lightflow itself (manual, on the host)

The dump goes back into the running lightflow-postgres container, with the server stopped; the raw data/postgres-db is never touched.

# 1. fetch the backup into /media/ssd/podman/lightflow/dataRestored/
#    (the live mirror; or --from-archive lightflow-YYYY-MM-DD.tar.gz,
#    which extracts into dataRestored/backup/ — adjust RESTORED below)
sudo backupctl --restore lightflow
RESTORED=/media/ssd/podman/lightflow/dataRestored

# 2. stop the server only: PostgreSQL keeps running
sudo -iu lightflow_svc systemctl --user stop lightflow-server.service

# 3. recreate an empty database (the container's local socket is trusted)
sudo -iu lightflow_svc podman exec lightflow-postgres dropdb -U lightflow lightflow
sudo -iu lightflow_svc podman exec lightflow-postgres createdb -U lightflow lightflow

# 4. load the dump, streamed from the root-only restore folder
sudo cat "${RESTORED}/lightflow.dump" | sudo -iu lightflow_svc \
  podman exec -i lightflow-postgres \
  pg_restore --no-owner --exit-on-error -U lightflow -d lightflow

# 5. start the server again: the backup run caught "running" in the dump
#    is marked failed at startup
sudo -iu lightflow_svc systemctl --user start lightflow-server.service

# 6. put the scripts back through the container, so they keep its uid
sudo cat "${RESTORED}/scripts.tar.gz" | sudo -iu lightflow_svc \
  podman exec -i lightflow-server tar -xzf - -C /

# 7. clean up
sudo rm -rf "${RESTORED}"

Use the database name and user from postgres.env if they differ from lightflow. Reinstall any extra package from the Python packages page.