Generated banner for Reconciler Logic
Generated logo for Reconciler Logic

Reconciler Logic

Grafana LabsOfficialPublished by Grafana Labs

  • Skill
  • Tool integration
  • Cloud and DevOps
  • Apache-2.0
  • No issues found

From the author

Implement reconcilers and watchers for grafana-app-sdk apps — write `TypedReconciler[*MyKind]` reconcile functions, apply generation-based skip patterns, do conflict-safe status updates via `resource.UpdateObject`, configure `BasicReconcileOptions` (namespace, label/field filters, finalizer management), use `Watcher` for event-style handling, reconcile `UnmanagedKinds` (resources your app doesn't own), and register the whole thing in `app.go`. Use when writing a reconciler, implementing the reconcile loop, adding async business logic, handling create/update/delete events, processing resource state changes, scheduling periodic resyncs with `RequeueAfter`, picking between Watcher and Reconciler, or wiring a controller into `app.go` — even when the user says "process this resource", "handle X events", or "write a controller" without saying "reconciler".
— the description by Grafana Labs

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides the patterns and procedures for implementing the asynchronous business-logic layer in grafana-app-sdk applications. It covers the creation of TypedReconciler functions, managing resource state changes, handling event-driven updates, and configuring informer options. It emphasizes safe status updates using resource.UpdateObject to prevent race conditions and provides guidance on managing finalizers and periodic resyncs to ensure system consistency.

What it does

  • Defines TypedReconciler patterns for strongly-typed resource processing.
  • Implements conflict-safe status updates using resource.UpdateObject.
  • Configures BasicReconcileOptions for namespace, label, and field filtering.
  • Manages resource finalizers for clean deletion of external dependencies.
  • Schedules periodic resyncs and async polling using RequeueAfter.
  • Wires controllers into the application registration process.

When to use it

  • When building the core business logic for a custom Kubernetes operator.
  • When you need to reconcile resources that your application does not own.
  • When handling complex create, update, or delete event cycles.
  • When you need to poll external systems asynchronously and update resource status.
  • When you need to filter which resources trigger reconciliation events.

Advantages

  • Prevents 409 conflict errors during status updates.
  • Reduces unnecessary processing with generation-based skip patterns.
  • Provides a structured, type-safe approach to operator development.
  • Simplifies finalizer management through opinionated defaults.

Usage examples

“How do I implement a reconciler for my custom resource MyKind?”
You define a struct embedding operator.TypedReconciler[*v1alpha1.MyKind], wire a ReconcileFunc to handle the logic, and register it in app.go using AppManagedKind.
“How can I avoid 409 conflicts when updating status?”
Use the resource.UpdateObject function, which fetches the latest object version before applying your update, ensuring atomic status changes.
“How do I make my reconciler poll an external system every 30 seconds?”
Return an operator.ReconcileResult with the RequeueAfter field set to 30 * time.Second within your reconcile function.
“How do I filter events so my reconciler only processes resources in the prod namespace?”
Configure the BasicReconcileOptions with the Namespace field set to 'prod' when registering the reconciler in your app configuration.

Install

  1. Download the skill as a ZIP:

    https://autogineer.com/agent-skills/grafana/reconciler-logic/download
  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip reconciler-logic.zip -d .agents/skills

Documentation by Grafana Labs

Reconciler Logic

Reconcilers are the async business-logic layer of a grafana-app-sdk app. The SDK enqueues a reconcile event when a resource is created, updated, or deleted; the reconciler observes the current state and drives the system toward the desired state.

Common Workflows

Implementing a new reconciler end-to-end
# 1. Generate operator stubs for a standalone app
grafana-app-sdk project component add operator

# 2. Implement the ReconcileFunc — see § TypedReconciler below for the pattern

# 3. Register the reconciler in app.go (see references/registration.md)

# 4. Generate, build, and verify it runs
grafana-app-sdk generate
go build ./...
go run ./cmd/operator   # tail logs — reconcile entries should appear when you kubectl-apply a resource

If the operator starts but no reconcile events fire when you create a resource:

  • Check BasicReconcileOptions.Namespace matches the resource's namespace

  • Check BasicReconcileOptions.LabelFilters / FieldSelectors — most "no events" issues are filter mismatches (kubectl get <resource> -o yaml to see labels)

  • Confirm the reconciler was attached to the right (latest) version of the kind

TypedReconciler — preferred pattern

operator.TypedReconciler handles type assertion and provides a strongly-typed ReconcileFunc:

type MyKindReconciler struct {
    operator.TypedReconciler[*v1alpha1.MyKind]
    client resource.Client
}

func NewMyKindReconciler(client resource.Client) *MyKindReconciler {
    r := &MyKindReconciler{client: client}
    r.ReconcileFunc = r.reconcile  // wire the typed func
    return r
}

func (r *MyKindReconciler) reconcile(
    ctx context.Context,
    req operator.TypedReconcileRequest[*v1alpha1.MyKind],
) (operator.ReconcileResult, error) {
    obj := req.Object

    // Skip if already reconciled this generation
    if obj.GetGeneration() == obj.Status.LastObservedGeneration &&
       req.Action != operator.ReconcileActionDeleted {
        return operator.ReconcileResult{}, nil
    }

    log := logging.FromContext(ctx).With("name", obj.GetName(), "namespace", obj.GetNamespace())
    log.Info("reconciling", "action", operator.ResourceActionFromReconcileAction(req.Action))

    if req.Action == operator.ReconcileActionDeleted {
        return operator.ReconcileResult{}, nil
    }

    // ... business logic ...

    // Atomic status update — see § Status updates below
    _, err := resource.UpdateObject(ctx, r.client, obj.GetStaticMetadata().Identifier(),
        func(obj *v1alpha1.MyKind, _ bool) (*v1alpha1.MyKind, error) {
            obj.Status.LastObservedGeneration = obj.GetGeneration()
            obj.Status.State = "Ready"
            return obj, nil
        },
        resource.UpdateOptions{Subresource: "status"},
    )
    return operator.ReconcileResult{}, err
}

ReconcileAction values: ReconcileActionCreated, ReconcileActionUpdated, ReconcileActionDeleted, ReconcileActionResynced.

To requeue after a delay (e.g. polling an external system):

return operator.ReconcileResult{RequeueAfter: 10 * time.Second}, nil

Status updates with resource.UpdateObject

Always use resource.UpdateObject for status writes — it fetches the latest version before applying your update function, avoiding 409 Conflict errors when multiple reconcile events race:

_, err := resource.UpdateObject(ctx, r.client, identifier,
    func(obj *v1alpha1.MyKind, exists bool) (*v1alpha1.MyKind, error) {
        obj.Status.LastObservedGeneration = obj.GetGeneration()
        obj.Status.State = "Ready"
        obj.Status.Message = ""
        return obj, nil
    },
    resource.UpdateOptions{Subresource: "status"},
)

Do not use client.Update for status — it sends the full object and races with spec changes made by users.

Generation-based skip

Check LastObservedGeneration at the top of the reconcile function to avoid re-processing unchanged resources:

if obj.GetGeneration() == obj.Status.LastObservedGeneration {
    return operator.ReconcileResult{}, nil
}

ReconcileOptions

Control informer behavior via BasicReconcileOptions on the AppManagedKind entry:

{
    Kind:       mykindv1alpha1.MyKindKind(),
    Reconciler: reconciler,
    ReconcileOptions: simple.BasicReconcileOptions{
        Namespace:      "my-namespace",          // watch one namespace; default is all
        LabelFilters:   []string{"env=prod"},    // only reconcile matching resources
        FieldSelectors: []string{"status.phase=Running"},
        UsePlain:       false,                   // false = wrap in OpinionatedReconciler (default; manages finalizers)
    },
},

UsePlain: false (the default) wraps your reconciler in OpinionatedReconciler, which manages finalizers automatically so the SDK can guarantee clean deletion.

References

  • references/watchers.md — Watcher alternative (event-style Add/Update/Delete callbacks) + decision matrix for watcher vs reconciler

  • references/unmanaged-kinds.md — UnmanagedKinds for reconciling resources your app doesn't own, with UseOpinionated: false guidance and common failure modes

  • references/registration.md — full app.go wiring (client setup, multi-version registration, ValidateManifest) + common failure modes

External resources

Files

references/informer-configuration.md

Informer Configuration

Reconcilers and Watchers receive events from an Informer, and both together are managed by an informer controller. The configuration of this controller created by simple.App is done in simple.AppConfig.InformerConfig.

InformerConfig

simple.AppConfig.InformerConfig is of type simple.AppInformerConfig, which is defined as:

// AppInformerConfig contains configuration for the App's internal operator.InformerController
type AppInformerConfig struct {
	// InformerOptions are the options for the informer.
	InformerOptions operator.InformerOptions
	// RetryPolicy is the policy for retrying events.
	RetryPolicy operator.RetryPolicy
	// RetryDequeuePolicy is the policy for dequeuing events.
	RetryDequeuePolicy operator.RetryDequeuePolicy
	// FinalizerSupplier is used to generate the finalizer for the kind.
	FinalizerSupplier operator.FinalizerSupplier
	// InProgressFinalizerSupplier is used to generate the "in-progress" finalizer used by opinionated adds,
	// before the "normal" finalizer (provided by FinalizerSupplier) is applied when the add completes successfully.
	// By default, this is "<app name>-wip"
	InProgressFinalizerSupplier operator.FinalizerSupplier
	// InformerSupplier can be set to specify a function for creating informers for kinds.
	// If left unset, DefaultInformerSupplier will be used.
	InformerSupplier InformerSupplier
}

operator.InformerOptions is defined as

// InformerOptions are generic options for all Informer implementations.
type InformerOptions struct {
	// ListWatchOptions are the options for filtering the watch based on namespace and other compatible filters.
	ListWatchOptions ListWatchOptions
	// CacheResyncInterval is the interval at which the informer will emit CacheResync events for all resources in the cache.
	// This is distinct from a full resync, as no information is fetched from the API server.
	// Changes to this value after run() is called will not take effect.
	// An empty value will disable cache resyncs.
	CacheResyncInterval time.Duration
	// EventTimeout is the timeout for an event to be processed.
	// If an event is not processed within this timeout, it will be dropped.
	// The timeout cannot be larger than the cache resync interval, if it is,
	// the cache resync interval will be used instead.
	// An empty value will disable event timeouts.
	EventTimeout time.Duration
	// ErrorHandler is called if the informer encounters an error which does not stop the informer from running,
	// but may stop it from processing a given event.
	ErrorHandler func(context.Context, error)
	// HealthCheckIgnoreSync will set the KubernetesBasedInformer HealthCheck to return ok once the informer is started,
	// rather than waiting for the informer to finish with its initial list sync.
	// You may want to set this to `true` if you have a particularly long initial sync period and don't want readiness checks failing.
	HealthCheckIgnoreSync bool
	// UseWatchList if turned on instructs the reflector to open a stream to bring data from the API server.
	// Streaming has the primary advantage of using fewer server's resources to fetch data.
	//
	// The old behavior establishes a LIST request which gets data in chunks.
	// Paginated list is less efficient and depending on the actual size of objects
	// might result in an increased memory consumption of the APIServer.
	//
	// Defaults to false. Requires Kubernetes 1.27+ when enabled.
	// See https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/3157-watch-list#design-details
	UseWatchList bool
	// WatchListPageSize is the requested chunk size for paginated LIST operations.
	// This significantly reduces memory usage when watching large numbers of objects (>10K).
	// Recommended values: 5000-10000 for most use cases.
	// Note: This only affects traditional LIST operations. It does NOT apply to watch-list streaming (UseWatchList).
	// An empty value (0) will use client-go's default pagination behavior based on resource version.
	WatchListPageSize int64
	// MaxConcurrentWorkers is the maximum number of concurrent workers for event processing in ConcurrentInformer.
	// Each worker maintains a queue of events which are processed sequentially.
	// Events for a particular object are assigned to the same worker to maintain in-order delivery per object.
	// An empty value (0) will use the default of 10 workers.
	MaxConcurrentWorkers uint64
}

Error Handling

If AppInformerConfig.InformerOptions.ErrorHandler is defined, it will be called prior to requeuing an error returned from a watcher or reconciler. By default, this is a function which logs the error to the logger from the context.

Retries and Dequeuing

simple.App internally manages retries and dequeueing of reconcile events. If a watcher or reconciler call returns an error, it is automatically retried based on the RetryPolicy. When a new event for the same resource arrives while a retry is still in the queue, the retry may be dequeued per the RetryDequeuePolicy.

RetryPolicy is defined as

// RetryPolicy is a function that defines whether an event should be retried, based on the error and number of attempts.
// It returns a boolean indicating whether another attempt should be made, and a time.Duration after which that attempt should be made again.
type RetryPolicy func(err error, attempt int) (bool, time.Duration)

If unspecified, AppInformerConfig.RetryPolicy defaults to an exponential backoff retry policy (operator.DefaultRetryPolicy) that retries a total of five times before giving up.

operator.ExponentialBackoffRetryPolicy(initialDelay time.Duration, maxAttempts int) can be used to create an arbitrary exponential backoff retry policy.

Custom retry policies may want to interrogate the supplied error to determine if a retry should be performed.

Dequeuing existing retries is managed with AppInformerConfig.RetryDequeuePolicy. A RetryDequeuePolicy is a function defined as:

// RetryDequeuePolicy is a function that defines when a retry should be dequeued when a new action is taken on a resource.
// It accepts information about the new action being taken, and information about the current queued retry,
// and returns `true` if the retry should be dequeued.
// A RetryDequeuePolicy may be called multiple times for the same action, depending on the number of pending retries for the object.
type RetryDequeuePolicy func(newAction ResourceAction, newObject resource.Object, retryAction ResourceAction, retryObject resource.Object, retryError error) bool

If unspecified, it uses the default (operator.OpinionatedRetryDequeuePolicy) that is defined as

// OpinionatedRetryDequeuePolicy is a RetryDequeuePolicy which has the following logic:
// 1. If the newAction is a delete, dequeue the retry
// 2. If the newAction and retryAction are different, keep the retry (for example, a queued create retry, and a received update action)
// 3. If the generation of newObject and retryObject is the same, keep the retry
// 4. Otherwise, dequeue the retry
var OpinionatedRetryDequeuePolicy = func(newAction ResourceAction, newObject resource.Object, retryAction ResourceAction, retryObject resource.Object, _ error) bool {
	if newAction == ResourceActionDelete {
		return true
	}
	if newAction != retryAction {
		return false
	}
	if newObject.GetGeneration() == retryObject.GetGeneration() {
		return false
	}
	return true
}

Finalizer Suppliers

Both the OpinionatedReconciler and OpinionatedWatcher manage finalizers, and the names of the finalizers are supplied by AppInformerConfig.FinalizerSupplier and AppInformerConfig.InProgressFinalizerSupplier.

An operator.FinalizerSupplier is defined as

// FinalizerSupplier represents a function that creates string finalizer from provider schema.
type FinalizerSupplier func(sch resource.Schema) string

By default, operator.DefaultFinalizerSupplier and operator.InProgressFinalizerSupplier are used, defined as:

// DefaultFinalizerSupplier crates finalizer following to pattern `operator.{version}.{kind}.{group}`.
func DefaultFinalizerSupplier(sch resource.Schema) string {
	return fmt.Sprintf("operator.%s.%s.%s", sch.Version(), sch.Kind(), sch.Group())
}

func InProgressFinalizerSupplier(sch resource.Schema) string {
	return fmt.Sprintf("wip.%s.%s.%s", sch.Version(), sch.Kind(), sch.Group())
}

The in-progress finalizer is added upon event ingestion by an OpinionatedReconciler or OpinionatedWatcher if no finalizer is present, and is replaced by the normal finalizer when the downstream request succeeds.

InformerSupplier

InformerSupplier is defined as

// InformerSupplier is a function which creates an operator.Informer for a kind, given a ClientGenerator and ListWatchOptions
type InformerSupplier func(
	kind resource.Kind, clients resource.ClientGenerator, options operator.InformerOptions,
) (operator.Informer, error)

and uses the following default if not overridden:

// DefaultInformerSupplier is a default InformerSupplier function which creates a basic operator.KubernetesBasedInformer.
// Note: This supplier does NOT respect WatchListPageSize or UseWatchList configuration.
// For memory-optimized informers that support these options, use OptimizedInformerSupplier instead.
var DefaultInformerSupplier = func(
	kind resource.Kind, clients resource.ClientGenerator, options operator.InformerOptions,
) (operator.Informer, error) {
	client, err := clients.ClientFor(kind)
	if err != nil {
		return nil, err
	}

	inf, err := operator.NewKubernetesBasedInformer(kind, client, options)
	if err != nil {
		return nil, err
	}

	return operator.NewConcurrentInformerFromOptions(inf, options)
}

It is used when simple.App creates a new Informer for a kind if a watcher or reconciler are configured for it. This allows the user to swap in different Informer implementations if they wish. The operator package has an alternate informer that can use a custom (and therefore not in-memory like the KubernetesBasedInformer) cache, using NewCustomCacheInformer, and an implementation of this using memcached as the storage with NewMemcachedInformer.

By default, the KubernetesBasedInformer, which uses kubernetes' cache.NewSharedIndexInformer under the hood, is wrapped in operator.NewConcurrentInformerFromOptions, which allows for concurrent event processing (as opposed to the standard kubernetes informer logic, where all events are placed in a queue and processed one by one). This concurrency guarantees that events for the same resource will be processed in sequence by the same goroutine, but otherwise will spread informer events across a configured number of goroutines to allow for faster processing of events, particularly at startup, when all resources currently in the APIServer must be processed before the informer will begin processing new events.

OptimizedInformerSupplier

simple.OptimizedInformerSupplier is an alternative to DefaultInformerSupplier that fully respects the WatchListPageSize and UseWatchList options in InformerOptions. The default supplier ignores these settings. Use it when watching large numbers of objects (>10K) or in memory-constrained environments:

simple.AppConfig{
    InformerConfig: simple.AppInformerConfig{
        InformerSupplier: simple.OptimizedInformerSupplier,
        InformerOptions: operator.InformerOptions{
            UseWatchList:      true,  // streaming, requires Kubernetes 1.27+
            WatchListPageSize: 5000,  // chunk size for paginated LIST fallback
        },
    },
    // ...
}

OptimizedInformerSupplier is marked experimental and may change in future releases.

Health Checks

By default, the health check returned by an Informer will not return healthy until all start-up events have been processed. While the ConcurrentInformer makes this faster, there may be scenarios where this start-up time is still too lengthy and having the pod be unhealthy for that start-up duration will cause it to be killed. In that case, AppInformerConfig.InformerOptions.HealthCheckIgnoreSync can be set to true to have the health check return healthy so long as the informer is running.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by
Grafana Labs
Publisher
Grafana Labs (Official)
Source
grafana/skills · skills/grafana-app-sdk/reconciler-logic · commit 8fb72a5
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Grafana Labs or Grafana Labs.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download reconciler-logic.zip