Flux

Flux

Overview

Flux is a tool that ensures a cluster stays in synchronization with a Git repository. It could be any Git repository, be it Forgejo, GitHub, git daemon, or even a bare repo over SSH.

The reason to implement this earlier rather than later is that it can:

1. Declaratively manage Helm charts (see Helm and Flux Integration for a worked example)
2. Transparently handle SOPS encrypted secrets

• It is in fact quite necessary to use SOPS with Flux, otherwise you would need some out-of-band process for managing secrets. By registering an Age key in the cluster, Flux can use that so that if secrets change it will just decrypt them in an ordinary way. See Secrets Management for the full setup.
• Use one age key per cluster.
  • SOPS supports multiple recipients, so encrypt each secret to multiple age keys at once.

3. Reconcile state

• Container images
  • Check for updates
  • Apply, rollout, fallback
    • Logging (use loki here)
  • Update manifest with current image
• Helm charts
  • Check for updates
  • Apply, rollout etc.
  • Use semver ranges in HelmRelease.spec.chart.spec.version (e.g. ^1.0.0) to auto-upgrade within a major version
  • Flux updates the running release, but does not commit the resolved version back to Git — use Renovate if you want manifest-level pinning
• Pulls from git every few minutes
  • Pulls change, rollout etc.

4. Drift

• Changes to a cluster will be reverted and pulled back in synchronization with Git.
  • I think this is arguably the most valuable feature of all. Even though getting updates from Git seems valuable and undoing changes somebody applied seems undesirable, Enforcing lockstep with a declarative configuration means that everybody knows exactly how the current setup works and it can be reproduced.

5. Notifications

• The notification controller can push alerts to Slack, Microsoft Teams, Discord, or any other arbitrary webhook when reconciliation succeeds or fails. So you get visibility without the pain of something like Vercel.

6. Health Checks

• Flux can wait for one resource to be ready before deploying the next. You don’t have to deal with explaining to the container how to apply the latest CI, CD. Even if Kubernetes does a lot for you, this is finished. You just add it and you’re done. Call it a day.

Bootstrap

Note

After performing the bootstrap, it is necessary to pull from the remote repository. For whatever reason, it will update the remote not the local copy

Pre-Requisites

One must have k0s running a cluster, flux will do the rest

Local Files

Use flux bootstrap git to bootstrap from any local or self-hosted Git repository. You’ll need a bare repo accessible over SSH or a local Git server.

Bare Repo over SSH

# Create a bare repo from your working repo
git clone --bare /path/to/your/repo /path/to/repos/my-repo.git

# Add it as a remote in your working repo
cd /path/to/your/repo
git remote add local /path/to/repos/my-repo.git

# Bootstrap Flux against it
flux bootstrap git \
  --url=ssh://localhost/path/to/repos/my-repo.git \
  --branch=main \
  --private-key-file=~/.ssh/id_ed25519 \
  --path=clusters/my-cluster

Local Git Daemon (no SSH key needed)

# Serve your repo over git protocol
git daemon --rw --base-path=/path/to/repos --export-all

# Bootstrap against it
flux bootstrap git \
  --url=git://localhost/my-repo.git \
  --branch=main \
  --path=clusters/my-cluster

Notes

• The --path flag scopes the cluster sync to a specific directory in the repo, allowing multiple clusters from one repo.
• Bootstrap is idempotent — safe to run as many times as needed.
• After bootstrap, all cluster operations (including Flux upgrades) can be done via git push.

Official docs: Flux bootstrap for Git servers

---

Forgejo

Forgejo is a fork of Gitea. Flux supports Gitea natively via flux bootstrap gitea, and Forgejo is backward compatible with this command.

Token Auth (HTTPS)

export GITEA_TOKEN=<your-forgejo-pat>

flux bootstrap gitea \
  --token-auth \
  --hostname=https://forgejo.example.com:3000 \
  --owner=<user-or-org> \
  --repository=fleet-infra \
  --branch=main \
  --path=clusters/my-cluster \
  --personal

SSH Auth

For existing repos, use flux bootstrap git with an SSH key:

flux bootstrap git \
  --url=ssh://git@forgejo.example.com/<org>/<repository> \
  --branch=main \
  --private-key-file=<path/to/private.key> \
  --password=<key-passphrase> \
  --path=clusters/my-cluster

Self-Signed Certificates

flux bootstrap gitea \
  --hostname=https://forgejo.example.com:3000 \
  --owner=<user> \
  --repository=fleet-infra \
  --branch=main \
  --path=clusters/my-cluster \
  --personal \
  --ca-file=./forgejo-ca.crt

Notes

• Forgejo requires HTTPS for token auth — flux bootstrap gitea will reject plain HTTP.
• For HTTP-only Forgejo instances, use flux bootstrap git with --allow-insecure-http=true --token-auth=true.
• The Forgejo PAT is stored as a Kubernetes Secret named flux-system in the flux-system namespace.
• SSH deploy keys are linked to the PAT — if the PAT is revoked, the deploy key stops working.

Official docs: Flux bootstrap for Gitea

---

GitHub

Commands

Flux

flux create source git my-app \
  --url=ssh://git@github.com/myorg/private-repo.git \
  --branch=main \
  --secret-ref=my-ssh-key

Kubectl

kubectl create secret generic my-ssh-key \
  --from-file=identity=~/.ssh/id_ed25519 \
  --from-file=known_hosts=~/.ssh/known_hosts

Full Bootstrap

export GITHUB_TOKEN=<your-pat>

flux bootstrap github \
  --owner=<org-or-user> \
  --repository=fleet-infra \
  --branch=main \
  --path=clusters/my-cluster \
  --personal

Notes

• If the repository doesn’t exist, Flux creates it as private by default.
• Use --token-auth=false to use SSH deploy keys instead of storing the PAT in-cluster.
• Fine-grained PATs need “Contents” read/write and “Administration” read/write permissions.

Official docs: Flux bootstrap for GitHub

Secrets

Overview

One must generate an Age key and register it as a secret in the Kubernetes cluster. If one has multiple clusters, one should have a separate age key for each cluster as Flux is cluster scoped. See Secrets Management for the full walkthrough.

So on you can run a command approximately similar to this:

age-keygen | tee kubectl create secret generic sops-age \
  --namespace=flux-system \
  --from-file=age.agekey=/dev/stdin

Store the key in KeePass, and then the SOPS encrypted secrets in the repository will be transparently dealt with by Flux.

One can also achieve this in typescript:

import { generateIdentity, identityToRecipient } from "age-encryption";

const secretKey = await generateIdentity(); // AGE-SECRET-KEY-1...
const publicKey = await identityToRecipient(secretKey); // age1...

// Now you can handle them however you want
const keyPair = {
  secretKey, // store in KeePass / cluster secret
  publicKey, // put in .sops.yaml
};

Image Automation

Flux can scan container registries for new tags, select the latest according to a policy, and commit the updated image reference back to Git. This requires two extra controllers that are not installed by default.

Bootstrap with Image Controllers

Re-bootstrap with --components-extra:

flux bootstrap github \
  --components-extra=image-reflector-controller,image-automation-controller \
  --owner=RyanGreenup \
  --repository=kubernetes-template \
  --branch=main \
  --path=clusters/vale \
  --read-write-key \
  --personal

--read-write-key is essential — the automation controller needs push access to commit image updates back to Git.

Three Resources

The pipeline has three resources, which form a chain:

ImageRepository  →  ImagePolicy  →  ImageUpdateAutomation
 (scan registry)    (select tag)     (commit to Git)

1. ImageRepository — what to scan

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
  name: podinfo
  namespace: flux-system
spec:
  image: ghcr.io/stefanprodan/podinfo
  interval: 5m
  # For private registries:
  # secretRef:
  #   name: regcred        # kubernetes.io/dockerconfigjson type
  exclusionList:
    - "^.*\\.sig$"

• .spec.image — registry address without scheme (docker.io/library/nginx, ghcr.io/org/app)
• .spec.interval — how often to poll
• .spec.secretRef — for private registries, reference a kubernetes.io/dockerconfigjson secret
• .spec.provider — generic (default), aws, azure, or gcp for native cloud auth
• .spec.exclusionList — regex patterns for tags to skip (defaults to ["^.*\\.sig$"] to exclude cosign signatures)

2. ImagePolicy — which tag to select

Three strategies: semver, alphabetical, numerical.

Semver (most useful):

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: podinfo
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: podinfo
  policy:
    semver:
      range: ">=1.0.0 <2.0.0" # auto-update minor+patch, pin major to 1

Semver range cheat sheet:

Range	Meaning
^1.0.0	>=1.0.0 <2.0.0 — minor + patch within 1.x
~1.2.0	>=1.2.0 <1.3.0 — patch only within 1.2.x
5.0.x	patch only within 5.0
>=1.0.0 <2.0.0	explicit form of ^1.0.0
*	any version (dangerous)

Tag filtering (extract semver from complex tags like 6.0.0-alpine3.12):

spec:
  filterTags:
    pattern: '^(?P<semver>[0-9]+\.[0-9]+\.[0-9]+)-(alpine.*)'
    extract: "$semver"
  policy:
    semver:
      range: ">=6.0.0"

3. ImageUpdateAutomation — how to commit

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageUpdateAutomation
metadata:
  name: flux-system
  namespace: flux-system
spec:
  interval: 30m
  sourceRef:
    kind: GitRepository
    name: flux-system
  git:
    checkout:
      ref:
        branch: main
    commit:
      author:
        email: fluxcdbot@users.noreply.github.com
        name: fluxcdbot
      messageTemplate: |
        Automated image update

        {{range .Changed.Changes}}
        {{print .OldValue}} -> {{println .NewValue}}
        {{end}}
    push:
      branch: main # push to a different branch for PR workflows
  update:
    path: ./clusters/vale
    strategy: Setters

• .spec.update.strategy must be Setters — this tells the controller to look for marker comments in YAML files
• .spec.update.path — directory to scan for markers (scope it to avoid scanning the whole repo)
• .spec.policySelector — limit which ImagePolicies this automation considers (useful for excluding detection-only policies, see below)

Image Markers

The automation controller finds fields to update via inline JSON comments in your YAML manifests. Add these to Deployment specs, HelmRelease values, etc.

Plain Deployment:

spec:
  containers:
    - name: app
      image: ghcr.io/stefanprodan/podinfo:5.0.0 # {"$imagepolicy": "flux-system:podinfo"}

HelmRelease values (tag and name separated):

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: podinfo
spec:
  values:
    image:
      repository: ghcr.io/stefanprodan/podinfo # {"$imagepolicy": "flux-system:podinfo:name"}
      tag: 5.0.0 # {"$imagepolicy": "flux-system:podinfo:tag"}

Marker suffixes: :name (repository only), :tag (tag only), :digest (digest only), or none (full image:tag).

Auto-Update Minor, Alert on Major

Flux doesn’t have a single “update minor but alert on major” switch. Instead, combine two ImagePolicies with the notification controller.

Step 1 — Policy that auto-updates (pinned to current major):

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: myapp
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: myapp
  policy:
    semver:
      range: ">=1.0.0 <2.0.0" # auto-updates 1.x.y

Step 2 — Detection-only policy (wide range, labelled):

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: myapp-latest
  namespace: flux-system
  labels:
    purpose: detection-only
spec:
  imageRepositoryRef:
    name: myapp
  policy:
    semver:
      range: ">=1.0.0" # matches 2.0.0+ too

Step 3 — Exclude detection policy from automation:

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageUpdateAutomation
metadata:
  name: flux-system
  namespace: flux-system
spec:
  # ... (same as above)
  policySelector:
    matchExpressions:
      - key: purpose
        operator: NotIn
        values:
          - detection-only

Step 4 — Alert when the detection policy selects a new major:

apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Alert
metadata:
  name: major-version-alert
  namespace: flux-system
spec:
  providerRef:
    name: slack # or any provider, see Notifications section
  eventSources:
    - kind: ImagePolicy
      name: myapp-latest

When myapp-latest resolves to 2.0.0, Flux fires an event. The pinned myapp policy stays on 1.x.y and keeps the cluster on the old major. You review the alert and bump the range manually.

---

Helm Chart Auto-Updates

Helm chart version updates are handled by the source-controller and helm-controller — no extra components needed. This is separate from image automation.

HelmRepository

apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: bitnami
  namespace: flux-system
spec:
  interval: 5m
  url: https://charts.bitnami.com/bitnami

For OCI-based charts:

apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: podinfo
  namespace: flux-system
spec:
  type: oci
  interval: 5m
  url: oci://ghcr.io/stefanprodan/charts

HelmRelease with Semver Ranges

The key is .spec.chart.spec.version. Use a semver range instead of an exact version and Flux will automatically upgrade when a new matching chart version appears in the repository index.

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: podinfo
  namespace: default
spec:
  interval: 5m
  chart:
    spec:
      chart: podinfo
      version: ">=1.0.0 <2.0.0" # auto-update minor+patch
      sourceRef:
        kind: HelmRepository
        name: podinfo
        namespace: flux-system
      interval: 5m # how often to check for new chart versions
  # Rollback on failure
  upgrade:
    remediation:
      retries: 3
  values:
    replicaCount: 2

Semver range examples for charts:

Range	Effect
'6.5.*'	Patch only within 6.5
'^1.0.0'	Minor + patch within major 1
'>=4.0.0 <5.0.0'	Same as ^4.0.0
'*'	Any version — do not use in production

How It Works

1. The source-controller periodically fetches the HelmRepository index (controlled by HelmRepository.spec.interval)
2. It resolves the best chart version matching the semver range
3. If the resolved version differs from the current release, the helm-controller performs a Helm upgrade
4. The running release is updated, but the resolved version is not committed back to Git — the HelmRelease YAML always contains the range, not the pinned version

Note

If you need the exact resolved version pinned in Git (e.g. for audit trails), use Renovate alongside Flux. Renovate opens PRs to bump chart versions in your manifests.

Upgrade Remediation

Configure rollback behaviour when an upgrade fails:

spec:
  upgrade:
    remediation:
      retries: 3
      remediateLastFailure: true # rollback to last successful release
  rollback:
    cleanupOnFail: true
    timeout: 5m

---

Notifications

The notification-controller ships with the default Flux install. It captures events from all Flux controllers and routes them to external systems.

Provider — where to send alerts

Slack (bot token — recommended):

apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Provider
metadata:
  name: slack
  namespace: flux-system
spec:
  type: slack
  channel: flux-alerts
  address: https://slack.com/api/chat.postMessage
  secretRef:
    name: slack-token
---
apiVersion: v1
kind: Secret
metadata:
  name: slack-token
  namespace: flux-system
stringData:
  token: xoxb-YOUR-BOT-TOKEN

Generic webhook:

apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Provider
metadata:
  name: webhook
  namespace: flux-system
spec:
  type: generic
  address: https://example.com/webhook

Generic webhook with HMAC:

apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Provider
metadata:
  name: webhook-hmac
  namespace: flux-system
spec:
  type: generic-hmac
  address: https://example.com/webhook
  secretRef:
    name: hmac-secret
---
apiVersion: v1
kind: Secret
metadata:
  name: hmac-secret
  namespace: flux-system
stringData:
  token: "your-hmac-key"

Other supported types: discord, msteams, googlechat, pagerduty, opsgenie, grafana, alertmanager, webex, rocket, azuredevops, azureeventhub, datadog, lark, matrix, nats, telegram, and more.

Alert — what events to forward

apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Alert
metadata:
  name: on-call
  namespace: flux-system
spec:
  providerRef:
    name: slack
  eventSeverity: info # 'info' (all) or 'error' (failures only)
  eventSources:
    - kind: GitRepository
      name: "*"
    - kind: Kustomization
      name: "*"
    - kind: HelmRelease
      name: "*"
    - kind: ImagePolicy
      name: "*"

• name: '*' — wildcard, matches all objects of that kind
• matchLabels — filter by labels instead of name
• .spec.inclusionList / .spec.exclusionList — Go regex filters on the event message body

Receiver — inbound webhooks

Trigger immediate reconciliation from external events (e.g. DockerHub push, GitHub webhook):

apiVersion: notification.toolkit.fluxcd.io/v1
kind: Receiver
metadata:
  name: dockerhub
  namespace: flux-system
spec:
  type: dockerhub # also: github, gitlab, harbor, quay, nexus, gcr
  secretRef:
    name: webhook-token
  resources:
    - kind: ImageRepository
      name: myapp

---

Putting It All Together — Example

A complete working setup for an app called webapp with:

• Auto-update container images on minor/patch
• Alert on major version bumps
• Auto-update Helm chart within major version

clusters/vale/
├── flux-system/           # bootstrap output
├── image-automation.yaml  # ImageUpdateAutomation
├── image-policies.yaml    # ImageRepository + ImagePolicy resources
├── helm-releases.yaml     # HelmRepository + HelmRelease resources
└── notifications.yaml     # Provider + Alert resources

image-policies.yaml

---
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
  name: webapp
  namespace: flux-system
spec:
  image: ghcr.io/myorg/webapp
  interval: 5m
  exclusionList:
    - "^.*\\.sig$"
---
# Auto-updates minor+patch within current major
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: webapp
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: webapp
  policy:
    semver:
      range: ">=1.0.0 <2.0.0"
---
# Detection-only — fires alert when major 2 appears
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: webapp-latest
  namespace: flux-system
  labels:
    purpose: detection-only
spec:
  imageRepositoryRef:
    name: webapp
  policy:
    semver:
      range: ">=1.0.0"

image-automation.yaml

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageUpdateAutomation
metadata:
  name: flux-system
  namespace: flux-system
spec:
  interval: 30m
  sourceRef:
    kind: GitRepository
    name: flux-system
  git:
    checkout:
      ref:
        branch: main
    commit:
      author:
        email: fluxcdbot@users.noreply.github.com
        name: fluxcdbot
      messageTemplate: |
        Automated image update

        {{range .Changed.Changes}}
        {{print .OldValue}} -> {{println .NewValue}}
        {{end}}
    push:
      branch: main
  update:
    path: ./clusters/vale
    strategy: Setters
  policySelector:
    matchExpressions:
      - key: purpose
        operator: NotIn
        values:
          - detection-only

helm-releases.yaml

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: bitnami
  namespace: flux-system
spec:
  interval: 10m
  url: https://charts.bitnami.com/bitnami
---
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: redis
  namespace: default
spec:
  interval: 5m
  chart:
    spec:
      chart: redis
      version: ">=19.0.0 <20.0.0"
      sourceRef:
        kind: HelmRepository
        name: bitnami
        namespace: flux-system
      interval: 10m
  upgrade:
    remediation:
      retries: 3
  values:
    architecture: standalone

notifications.yaml

---
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Provider
metadata:
  name: slack
  namespace: flux-system
spec:
  type: slack
  channel: flux-alerts
  address: https://slack.com/api/chat.postMessage
  secretRef:
    name: slack-token
---
# Alert on major version detection
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Alert
metadata:
  name: major-version-alert
  namespace: flux-system
spec:
  providerRef:
    name: slack
  eventSources:
    - kind: ImagePolicy
      name: webapp-latest
---
# Alert on all Helm upgrade failures
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Alert
metadata:
  name: helm-errors
  namespace: flux-system
spec:
  providerRef:
    name: slack
  eventSeverity: error
  eventSources:
    - kind: HelmRelease
      name: "*"

References

• Flux installation overview
• Flux bootstrap command reference
• Flux bootstrap for generic Git servers
• Flux bootstrap for Gitea / Forgejo
• Flux bootstrap for GitHub
• Getting started with Flux
• Image update guide
• ImageRepository API
• ImagePolicy API
• ImageUpdateAutomation API
• HelmRelease API
• Notification controller
• Alert API