Flux – Toolkit Dev Guides

Flux: Using the GitOps Toolkit APIs with Go

Mon, 01 Jan 0001 00:00:00 +0000

While you can use the GitOps Toolkit APIs in a declarative manner with kubectl apply, we provide client library code for all our toolkit APIs that makes it easier to access them from Go.

Go Packages

The GitOps Toolkit Go modules and controllers are released by following the semver conventions.

The API schema definitions modules have the following dependencies:

The APIs can be consumed with the controller-runtime client.

source.toolkit.fluxcd.io

Download package

go get github.com/fluxcd/source-controller/api

Import package

import sourcev1 "github.com/fluxcd/source-controller/api/v1"

API Types

Name	Version
GitRepository	v1
HelmRepository	v1
HelmChart	v1
OCIRepository	v1
Bucket	v1
ExternalArtifact	v1

kustomize.toolkit.fluxcd.io

Download package

go get github.com/fluxcd/kustomize-controller/api

Import package

import kustomizev1 "github.com/fluxcd/kustomize-controller/api/v1"

API Types

Name	Version
Kustomization	v1

helm.toolkit.fluxcd.io

Download package

go get github.com/fluxcd/helm-controller/api

Import package

import helmv2 "github.com/fluxcd/helm-controller/api/v2"

API Types

Name	Version
HelmRelease	v2

notification.toolkit.fluxcd.io

Download package

go get github.com/fluxcd/notification-controller/api

Import package

import notificationv1b3 "github.com/fluxcd/notification-controller/api/v1beta3"

and for Receiver objects:

import notificationv1 "github.com/fluxcd/notification-controller/api/v1"

API Types

Name	Version
Receiver	v1
Provider	v1beta3
Alert	v1beta3

image.toolkit.fluxcd.io

Download package

go get github.com/fluxcd/image-reflector-controller/api
go get github.com/fluxcd/image-automation-controller/api

Import package

import (
 imagev1 "github.com/fluxcd/image-reflector-controller/api/v1beta2"
 autov1 "github.com/fluxcd/image-automation-controller/api/v1beta2"
)

API Types

Name	Version
ImageRepository	v1beta2
ImagePolicy	v1beta2
ImageUpdateAutomation	v1beta2

CRUD Example

Here is an example of how to create a Helm release, wait for it to install, then delete it:

package main

import (
 "context"
 "fmt"
 "time"

 apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
 "k8s.io/apimachinery/pkg/api/meta"
 metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 "k8s.io/apimachinery/pkg/runtime"
 "k8s.io/apimachinery/pkg/types"
 "k8s.io/apimachinery/pkg/util/wait"
 _ "k8s.io/client-go/plugin/pkg/client/auth"
 ctrl "sigs.k8s.io/controller-runtime"
 "sigs.k8s.io/controller-runtime/pkg/client"

 helmv2 "github.com/fluxcd/helm-controller/api/v2"
 apimeta "github.com/fluxcd/pkg/apis/meta"
 sourcev1 "github.com/fluxcd/source-controller/api/v1"
)

func main() {
 // register the GitOps Toolkit schema definitions
 scheme := runtime.NewScheme()
 _ = sourcev1.AddToScheme(scheme)
 _ = helmv2.AddToScheme(scheme)

 // init Kubernetes client
 kubeClient, err := client.New(ctrl.GetConfigOrDie(), client.Options{Scheme: scheme})
 if err != nil {
 panic(err)
 }

 // set a deadline for the Kubernetes API operations
 ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
 defer cancel()

 // create a Helm repository pointing to Bitnami
 helmRepository := &sourcev1.HelmRepository{
 ObjectMeta: metav1.ObjectMeta{
 Name: "bitnami",
 Namespace: "default",
 },
 Spec: sourcev1.HelmRepositorySpec{
 URL: "https://charts.bitnami.com/bitnami",
 Interval: metav1.Duration{
 Duration: 30 * time.Minute,
 },
 },
 }
 if err := kubeClient.Create(ctx, helmRepository); err != nil {
 fmt.Println(err)
 } else {
 fmt.Println("HelmRepository bitnami created")
 }

 // create a Helm release for nginx
 helmRelease := &helmv2.HelmRelease{
 ObjectMeta: metav1.ObjectMeta{
 Name: "nginx",
 Namespace: "default",
 },
 Spec: helmv2.HelmReleaseSpec{
 ReleaseName: "nginx",
 Interval: metav1.Duration{
 Duration: 5 * time.Minute,
 },
 Chart: helmv2.HelmChartTemplate{
 Spec: helmv2.HelmChartTemplateSpec{
 Chart: "nginx",
 Version: "8.x",
 SourceRef: helmv2.CrossNamespaceObjectReference{
 Kind: sourcev1.HelmRepositoryKind,
 Name: "bitnami",
 },
 },
 },
 Values: &apiextensionsv1.JSON{Raw: []byte(`{"service": {"type": "ClusterIP"}}`)},
 },
 }
 if err := kubeClient.Create(ctx, helmRelease); err != nil {
 fmt.Println(err)
 } else {
 fmt.Println("HelmRelease nginx created")
 }

 // wait for the a Helm release to be reconciled
 fmt.Println("Waiting for nginx to be installed")
 if err := wait.PollImmediate(2*time.Second, 1*time.Minute,
 func() (done bool, err error) {
 namespacedName := types.NamespacedName{
 Namespace: helmRelease.GetNamespace(),
 Name: helmRelease.GetName(),
 }
 if err := kubeClient.Get(ctx, namespacedName, helmRelease); err != nil {
 return false, err
 }
 return meta.IsStatusConditionTrue(helmRelease.Status.Conditions, apimeta.ReadyCondition), nil
 }); err != nil {
 fmt.Println(err)
 }

 // print the reconciliation status
 fmt.Println(meta.FindStatusCondition(helmRelease.Status.Conditions, apimeta.ReadyCondition).Message)

 // uninstall the release and delete the repository
 if err := kubeClient.Delete(ctx, helmRelease); err != nil {
 fmt.Println(err)
 }
 if err := kubeClient.Delete(ctx, helmRepository); err != nil {
 fmt.Println(err)
 }
 fmt.Println("Helm repository and release deleted")
}

For an example on how to build a Kubernetes controller that interacts with the GitOps Toolkit APIs see source-watcher.

Artifact SDK

The Artifact SDK provides comprehensive functionality for packaging, storing, serving, and processing Flux artifacts. It is the foundation for building 3rd-party controllers that manage ExternalArtifact resources.

Download package

go get github.com/fluxcd/pkg/artifact

Import packages

import (
 "github.com/fluxcd/pkg/artifact/config"
 "github.com/fluxcd/pkg/artifact/server"
 "github.com/fluxcd/pkg/artifact/storage"
 "github.com/fluxcd/pkg/artifact/digest"
)

Sub-package	Purpose
config	Flag binding and configuration for storage, server, retention, and digest options
server	HTTP file server with graceful shutdown for serving artifacts in-cluster
storage	Artifact lifecycle management — create, archive, verify, copy, GC
digest	Multi-algorithm digest computation (SHA256, SHA512, BLAKE3)

For a step-by-step guide on building a controller using the SDK, see Building ExternalArtifact Controllers.

Flux: Watching for source changes

Mon, 01 Jan 0001 00:00:00 +0000

In this guide you’ll be developing a Kubernetes controller with Kubebuilder that subscribes to GitRepository events and reacts to revision changes by downloading the artifact produced by source-controller.

Prerequisites

On your dev machine install the following tools:

go >= 1.22
kubebuilder >= 3.0
kind >= 0.22
kubectl >= 1.29

Install Flux

Install the Flux CLI with Homebrew on macOS or Linux:

brew install fluxcd/tap/flux

Create a cluster for testing:

kind create cluster --name dev

Verify that your dev machine satisfies the prerequisites with:

flux check --pre

Install source-controller on the dev cluster:

flux install \
--namespace=flux-system \
--network-policy=false \
--components=source-controller

Clone the sample controller

You’ll be using fluxcd/source-watcher as a template for developing your own controller. The source-watcher was scaffolded with kubebuilder init.

Clone the source-watcher repository:

git clone https://github.com/fluxcd/source-watcher
cd source-watcher
git checkout release/v1.3.x

Build the controller:

make

Run the controller

Port forward to source-controller artifacts server:

kubectl -n flux-system port-forward svc/source-controller 8181:80

Export the local address as SOURCE_CONTROLLER_LOCALHOST:

export SOURCE_CONTROLLER_LOCALHOST=localhost:8181

Run source-watcher locally:

make run

Create a Git source:

flux create source git test \
--url=https://github.com/fluxcd/flux2 \
--ignore-paths='/*,!/manifests' \
--tag=v2.2.0

The source-watcher will log the revision:

{"level":"info","ts":"2024-05-14T16:43:42.703+0200","msg":"New revision detected","controller":"gitrepository","controllerGroup":"source.toolkit.fluxcd.io","controllerKind":"GitRepository","GitRepository":{"name":"test","namespace":"flux-system"},"namespace":"flux-system","name":"test","reconcileID":"ef0fe80e-3952-4835-ae9d-01760c4eadde","revision":"v2.2.0@sha1:81606709114f6d16a432f9f4bfc774942f054327"}

Change the Git tag:

flux create source git test \
--url=https://github.com/fluxcd/flux2 \
--ignore-paths='/*,!/manifests' \
--tag=v2.3.0

And source-watcher will log the new revision:

{"level":"info","ts":"2024-05-14T16:51:33.499+0200","msg":"New revision detected","controller":"gitrepository","controllerGroup":"source.toolkit.fluxcd.io","controllerKind":"GitRepository","GitRepository":{"name":"test","namespace":"flux-system"},"namespace":"flux-system","name":"test","reconcileID":"cc0f83bb-b7a0-4c19-a254-af9962ae39cd","revision":"v2.3.0@sha1:658925c2c0e6c408597d907a8ebee06a9a6d7f30"}

The source-controller reports the revision under GitRepository.Status.Artifact.Revision in the format: <branch|tag>@sha1:<commit>.

How it works

The GitRepositoryWatcher controller does the following:

subscribes to GitRepository events
detects when the Git revision changes
downloads and extracts the source artifact
writes the extracted dir names to stdout

type GitRepositoryWatcher struct {
 client.Client
 HttpRetry int
 artifactFetcher *fetch.ArchiveFetcher
}

func (r *GitRepositoryWatcher) SetupWithManager(mgr ctrl.Manager) error {
 r.artifactFetcher = fetch.NewArchiveFetcher(
 r.HttpRetry,
 tar.UnlimitedUntarSize,
 os.Getenv("SOURCE_CONTROLLER_LOCALHOST"),
 )

 return ctrl.NewControllerManagedBy(mgr).
 For(&sourcev1.GitRepository{}, builder.WithPredicates(GitRepositoryRevisionChangePredicate{})).
 Complete(r)
}

// +kubebuilder:rbac:groups=source.toolkit.fluxcd.io,resources=gitrepositories,verbs=get;list;watch
// +kubebuilder:rbac:groups=source.toolkit.fluxcd.io,resources=gitrepositories/status,verbs=get

func (r *GitRepositoryWatcher) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
 log := ctrl.LoggerFrom(ctx)

 // get source object
 var repository sourcev1.GitRepository
 if err := r.Get(ctx, req.NamespacedName, &repository); err != nil {
 return ctrl.Result{}, client.IgnoreNotFound(err)
 }

 artifact := repository.Status.Artifact
 log.Info("New revision detected", "revision", artifact.Revision)

 // create tmp dir
 tmpDir, err := os.MkdirTemp("", repository.Name)
 if err != nil {
 return ctrl.Result{}, fmt.Errorf("failed to create temp dir, error: %w", err)
 }
 defer os.RemoveAll(tmpDir)

 // download and extract artifact
 if err := r.artifactFetcher.Fetch(artifact.URL, artifact.Digest, tmpDir); err != nil {
 log.Error(err, "unable to fetch artifact")
 return ctrl.Result{}, err
 }

 // list artifact content
 files, err := os.ReadDir(tmpDir)
 if err != nil {
 return ctrl.Result{}, fmt.Errorf("failed to list files, error: %w", err)
 }

 // do something with the artifact content
 for _, f := range files {
 log.Info("Processing " + f.Name())
 }

 return ctrl.Result{}, nil
}

To add the watcher to an existing project, copy the controller and the revision change predicate to your controllers dir:

In your main.go init function, register the Source API schema:

import (
 utilruntime "k8s.io/apimachinery/pkg/util/runtime"
 clientgoscheme "k8s.io/client-go/kubernetes/scheme"

 sourcev1 "github.com/fluxcd/source-controller/api/v1"
)

func init() {
 utilruntime.Must(clientgoscheme.AddToScheme(scheme))
 utilruntime.Must(sourcev1.AddToScheme(scheme)

 // +kubebuilder:scaffold:scheme
}

Start the controller in the main function:

func main() {

 if err = (&controllers.GitRepositoryWatcher{
 Client: mgr.GetClient(),
 HttpRetry: httpRetry,
 }).SetupWithManager(mgr); err != nil {
 setupLog.Error(err, "unable to create controller", "controller", "GitRepositoryWatcher")
 os.Exit(1)
 }

}

Note that the watcher depends on Flux runtime and Kubernetes controller-runtime:

require (
 github.com/fluxcd/pkg/runtime v0.47.1
 sigs.k8s.io/controller-runtime v0.18.2
)

That’s it! Happy hacking!

Building artifact generators

If you want to build a controller that generates artifacts (not just watches them), see Building ExternalArtifact Controllers for a guide on using the Flux Artifact SDK to create ExternalArtifact resources that can be consumed by kustomize-controller and helm-controller.

Flux: Building ExternalArtifact Controllers

Mon, 01 Jan 0001 00:00:00 +0000

In this guide you’ll learn how to build a Kubernetes controller that acts as a 3rd-party source of truth for Flux by creating ExternalArtifact resources. Your controller will use the Flux Artifact SDK ( github.com/fluxcd/pkg/artifact) to package, store, and serve artifacts that can be consumed by kustomize-controller and helm-controller.

Overview

The ExternalArtifact API (part of RFC-0012) allows 3rd-party controllers to expose artifacts in-cluster in the same way source-controller does. This means Flux Kustomization and HelmRelease resources can reference your custom source types via ExternalArtifact without any changes to the Flux core.

The Artifact SDK provides four sub-packages:

Package	Import Path	Purpose
config	`github.com/fluxcd/pkg/artifact/config`	Flag binding and configuration for storage, server, retention, and digest options
server	`github.com/fluxcd/pkg/artifact/server`	HTTP file server with graceful shutdown for serving artifacts in-cluster
storage	`github.com/fluxcd/pkg/artifact/storage`	Artifact lifecycle management — create, archive, verify, copy, GC
digest	`github.com/fluxcd/pkg/artifact/digest`	Multi-algorithm digest computation (SHA1,SHA256, SHA512, BLAKE3)

Prerequisites

On your dev machine install the following tools:

go >= 1.24
kubebuilder >= 4.0
kind >= 0.22
kubectl >= 1.31
Flux CLI >= 2.7

Install Flux

Create a cluster for testing:

kind create cluster --name dev

Install Flux with the ExternalArtifact feature gate enabled:

flux install \
 --namespace=flux-system \
 --network-policy=false \
 --components=source-controller,kustomize-controller,helm-controller

Enable the ExternalArtifact feature gate on kustomize-controller and helm-controller:

kubectl -n flux-system patch deployment kustomize-controller \
 --type='json' -p='[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--feature-gates=ExternalArtifact=true"}]'

kubectl -n flux-system patch deployment helm-controller \
 --type='json' -p='[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--feature-gates=ExternalArtifact=true"}]'

Reference implementation

The fluxcd/source-watcher repository contains a full reference implementation (branch v2) of an ArtifactGenerator controller that uses the ExternalArtifact API and SDK. Clone it to follow along:

git clone https://github.com/fluxcd/source-watcher
cd source-watcher
git checkout v2

SDK Quick Start

1. Add the dependency

go get github.com/fluxcd/pkg/artifact
go get github.com/fluxcd/source-controller/api

2. Configure the artifact server

Use config.Options to declare storage and server settings. The SDK provides flag binding with environment variable support out of the box:

import (
 "github.com/spf13/pflag"
 "github.com/fluxcd/pkg/artifact/config"
)

func main() {
 opts := &config.Options{}

 // Bind CLI flags for --storage-path, --storage-addr,
 // --storage-adv-addr, --artifact-retention-ttl,
 // --artifact-retention-records, --artifact-digest-algo.
 opts.BindFlags(pflag.CommandLine)
 pflag.Parse()
}

Available configuration flags and their defaults:

Flag	Env Var	Default	Description
`--storage-path`	`STORAGE_PATH`	`/data`	Directory where artifacts are stored
`--storage-addr`	`STORAGE_ADDRESS`	`:9090`	Address the artifact server binds to
`--storage-adv-addr`	`STORAGE_ADV_ADDR`	(auto)	In-cluster address advertised to clients
`--artifact-retention-ttl`	—	`1m`	Duration after which stale artifacts are GC’d
`--artifact-retention-records`	—	`2`	Max artifacts kept per source after GC
`--artifact-digest-algo`	—	`sha256`	Hashing algorithm for artifact digests

3. Initialize Storage

The storage.Storage type manages artifact tarballs on the local filesystem:

import (
 "github.com/fluxcd/pkg/artifact/config"
 "github.com/fluxcd/pkg/artifact/storage"
)

// Create storage from configuration options.
store, err := storage.New(opts)
if err != nil {
 panic(err)
}

4. Start the artifact file server

Start the HTTP file server after the controller manager is elected leader. The server exposes artifacts under the configured storage path and supports graceful shutdown via context cancellation:

import (
 "github.com/fluxcd/pkg/artifact/server"
)

// Start the artifact server after the controller-manager receives leadership.
go func() {
 <-mgr.Elected()
 if err := server.Start(ctx, opts); err != nil {
 setupLog.Error(err, "unable to start artifact server")
 }
}()

5. Create and archive artifacts

In your controller’s Reconcile function, use the storage API to create new artifacts, archive directories into tarballs, and set digest/size metadata:

import (
 metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 "github.com/fluxcd/pkg/apis/meta"
 "github.com/fluxcd/pkg/artifact/storage"
)

func (r *ArtifactGeneratorReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
 // Create a new artifact descriptor.
 artifact := store.NewArtifactFor(
 "MySource", // kind
 &mySourceObject.ObjectMeta, // metadata (namespace + name)
 revision, // e.g. "v1.0.0@sha256:abc123..."
 fmt.Sprintf("%s.tar.gz", hash),// filename
 )

 // Ensure the artifact directory exists.
 if err := store.MkdirAll(artifact); err != nil {
 return ctrl.Result{}, err
 }

 // Archive a directory into a tarball.
 // The filter excludes .git and other VCS directories.
 if err := store.Archive(&artifact, "/path/to/source/dir", nil); err != nil {
 return ctrl.Result{}, err
 }

 // At this point, artifact.Digest, artifact.Size, and
 // artifact.LastUpdateTime are automatically set by the SDK.
 return ctrl.Result{}, nil
}

6. Apply the ExternalArtifact status

After archiving, create or update the ExternalArtifact resource in the cluster. The ExternalArtifact status must contain the artifact metadata so that kustomize-controller and helm-controller can fetch and verify it:

import (
 sourcev1 "github.com/fluxcd/source-controller/api/v1"
 metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 "sigs.k8s.io/controller-runtime/pkg/client"
)

func (r *ArtifactGeneratorReconciler) reconcileExternalArtifact(ctx context.Context,
 name, namespace string, artifact meta.Artifact) error {

 ea := &sourcev1.ExternalArtifact{
 ObjectMeta: metav1.ObjectMeta{
 Name: name,
 Namespace: namespace,
 },
 }

 _, err := ctrl.CreateOrUpdate(ctx, r.Client, ea, func() error {
 // Set the artifact status.
 ea.Status.Artifact = &artifact
 // Mark the ExternalArtifact as ready.
 ea.Status.Conditions = []metav1.Condition{
 {
 Type: "Ready",
 Status: metav1.ConditionTrue,
 LastTransitionTime: metav1.Now(),
 Reason: "Succeeded",
 Message: fmt.Sprintf("stored artifact for revision %s", artifact.Revision),
 },
 }
 return nil
 })
 return err
}

7. Implement garbage collection

The SDK provides built-in garbage collection based on retention TTL and record count limits:

import (
 "time"
 "github.com/fluxcd/pkg/artifact/storage"
)

func (r *ArtifactGeneratorReconciler) garbageCollect(ctx context.Context, artifact meta.Artifact) error {
 // GarbageCollect removes stale artifacts based on the
 // configured retention TTL and max records.
 deleted, err := store.GarbageCollect(ctx, artifact, 5*time.Minute)
 if err != nil {
 return err
 }
 if len(deleted) > 0 {
 log.Info("garbage collected artifacts", "count", len(deleted))
 }
 return nil
}

8. Verify artifact integrity

At startup, verify that artifacts in storage have not been tampered with:

// Verify that the artifact on disk matches the expected digest.
if err := store.VerifyArtifact(artifact); err != nil {
 log.Error(err, "artifact integrity check failed")
 // Re-fetch or re-generate the artifact.
}

Consuming ExternalArtifacts

Once your controller creates ExternalArtifact resources, Flux users can reference them in Kustomization and HelmRelease resources.

With Kustomization

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
 name: my-app
 namespace: apps
spec:
 interval: 10m
 sourceRef:
 kind: ExternalArtifact
 name: my-app
 path: "./"
 prune: true

With HelmRelease

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
 name: my-chart
 namespace: apps
spec:
 interval: 10m
 releaseName: my-chart
 chartRef:
 kind: ExternalArtifact
 name: my-chart

Storage API Reference

The storage.Storage type provides the following operations:

Artifact Creation

Method	Description
`NewArtifactFor(kind, metadata, revision, fileName)`	Create a new `meta.Artifact` descriptor with URL
`MkdirAll(artifact)`	Create the artifact’s base directory
`Archive(artifact, dir, filter)`	Archive a directory to a tarball with digest computation
`AtomicWriteFile(artifact, reader, mode)`	Atomically write content to the artifact path
`Copy(artifact, reader)`	Atomically copy reader content to the artifact path
`CopyFromPath(artifact, path)`	Atomically copy a file to the artifact path

Artifact Verification

Method	Description
`ArtifactExist(artifact)`	Check if an artifact exists in storage
`VerifyArtifact(artifact)`	Verify artifact integrity against its digest
`Lock(artifact)`	Create a file lock for the artifact

Artifact Cleanup

Method	Description
`Remove(artifact)`	Remove a single artifact file
`RemoveAll(artifact)`	Remove the artifact’s entire directory
`RemoveAllButCurrent(artifact)`	Remove all files except the current artifact
`GarbageCollect(ctx, artifact, timeout)`	GC stale artifacts based on retention policy

Path and URL Helpers

Method	Description
`LocalPath(artifact)`	Secure local path of an artifact (relative to `BasePath`)
`SetArtifactURL(artifact)`	Set the HTTP URL on an artifact
`SetHostname(URL)`	Replace the hostname of a URL
`Symlink(artifact, linkName)`	Create or update a symlink for the artifact
`ArtifactPath(kind, ns, name, file)`	Generate an artifact path string
`ArtifactDir(kind, ns, name)`	Generate an artifact directory path string

Security Best Practices

When building 3rd-party controllers that generate ExternalArtifact resources, follow these security guidelines from RFC-0012:

Authentication & Authorization: Use serviceAccountName for workload identity, secretRef for long-lived credentials. Never cache credentials on disk or in-memory.
TLS Encryption: Use certSecretRef for custom CA certificates. Prefer Mutual TLS authentication. Never skip TLS verification.
Provenance & Integrity: Verify upstream artifacts using Sigstore Cosign or Notary Notation signatures. Prefer keyless verification with OIDC tokens.
Access Control: Expose a --no-cross-namespace-refs flag to restrict cross-namespace ExternalArtifact generation. Use Kubernetes owner references for garbage collection.
Least Privilege: Use a dedicated service account with minimal RBAC. Conform with the restricted pod security standard (no root, read-only rootfs).
Storage Integrity: At startup, verify all stored artifact checksums against the ExternalArtifact digests in the cluster.
Network Policies: Restrict artifact endpoint access to only kustomize-controller and helm-controller.

Policy Enforcement

Cluster administrators can restrict which controllers can create ExternalArtifact resources using ValidatingAdmissionPolicy:

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
 name: "trusted-external-artifacts"
spec:
 failurePolicy: Fail
 matchConstraints:
 resourceRules:
 - apiGroups: ["source.toolkit.fluxcd.io"]
 apiVersions: ["v1"]
 operations: ["CREATE", "UPDATE"]
 resources: ["externalartifacts"]
 validations:
 # Restrict the artifacts to be served only by trusted endpoints
 - expression: >
 !has(object.status.artifact) ||
 object.status.artifact.url.startsWith('http://my-controller.flux-system.svc.cluster.local./')
 # Restrict the artifact operations to trusted service accounts
 - expression: >
 request.userInfo.username == 'system:serviceaccount:flux-system:my-controller'

Flux: Advanced debugging

Mon, 01 Jan 0001 00:00:00 +0000

This guide covers more advanced debugging topics such as collecting runtime profiling data from GitOps Toolkit components.

As a user, this page normally should be a last resort, but you may be asked by a maintainer to share a collected profile to debug e.g. performance issues.

Pprof

The GitOps Toolkit components serve pprof runtime profiling data on their metrics HTTP server (default :8080).

Endpoints

Endpoint	Path
Index	`/debug/pprof/`
CPU profile	`/debug/pprof/profile`
Symbol	`/debug/pprof/symbol`
Trace	`/debug/pprof/trace`

Collecting a profile

To collect a profile, port-forward to the component’s metrics endpoint and collect the data from the endpoint of choice:

$ kubectl port-forward -n <namespace> deploy/<component> 8080
$ curl -Sk -v http://localhost:8080/debug/pprof/heap > heap.out

The collected profile can be analyzed using go, or shared with one of the maintainers.

Resource usage

As kubectl top gives a limited (and at times inaccurate) overview of resource usage, it is often better to make use of the Grafana metrics to gather insights. See Flux Prometheus metrics for a guide on how to visualize this data with a Grafana dashboard.

Flux – Toolkit Dev Guides

Flux: Using the GitOps Toolkit APIs with Go

Go Packages

source.toolkit.fluxcd.io

kustomize.toolkit.fluxcd.io

helm.toolkit.fluxcd.io

notification.toolkit.fluxcd.io

image.toolkit.fluxcd.io

CRUD Example

Artifact SDK

Flux: Watching for source changes

Prerequisites

Install Flux

Clone the sample controller

Run the controller

How it works

Building artifact generators

Flux: Building ExternalArtifact Controllers

Overview

Prerequisites

Install Flux

Reference implementation

SDK Quick Start

1. Add the dependency

2. Configure the artifact server

3. Initialize Storage

4. Start the artifact file server

5. Create and archive artifacts

6. Apply the ExternalArtifact status

7. Implement garbage collection

8. Verify artifact integrity

Consuming ExternalArtifacts

With Kustomization

With HelmRelease

Storage API Reference

Artifact Creation

Artifact Verification

Artifact Cleanup

Path and URL Helpers

Security Best Practices

Policy Enforcement

Further Reading

Flux: Advanced debugging

Pprof

Endpoints

Collecting a profile

Resource usage