NEBUACLOUD
DashboardpricingLabsAgenciesNebuacloud for BusinessDocs

template.kicker

Operator Installation Guide

How to install and configure nebua-operator, the optional in-cluster operator for self-healing, policy enforcement, and add-on lifecycle

Nebua Operator Installation Guide

nebua-operator is the optional in-cluster Kubernetes operator for advanced NebuaCloud platform features. Unlike nebua-agent, which runs on the host and connects outbound to NebuaCloud, the operator runs entirely inside the cluster using an in-cluster service account and reconciles Kubernetes-native resources and NebuaCloud CRDs directly. See Nebua Runtime Components for how it fits alongside the agent and gateway.

Install nebua-operator when you need any of:

  • self-healing and bounded workload remediation (crash loops, image pull failures, unavailable replicas, node pressure, stuck PVCs, networking symptoms)
  • policy enforcement through NebuaPolicy and admission policy providers
  • backup orchestration through NebuaBackupPlan
  • GitOps enforcement and drift control
  • add-on install, upgrade, health, drift detection, and repair

The platform still supports basic cluster management through nebua-agent alone — the operator is optional.

Prerequisites

  • A working kubectl context with cluster-admin (or equivalent) permissions on the target cluster
  • Cluster reachable from wherever you run the installer (the installer applies manifests through your local kubectl)
  • Outbound HTTPS access from inside the cluster to github.com/objects.githubusercontent.com — the operator Deployment's bundle-fetch initContainer downloads the compiled operator bundle at pod startup, same as nebua-agent/nebua-gateway do on their hosts
  • A namespace for the operator, default nebua-system (created automatically)

Install

There are three ways to install nebua-operator, in order of convenience:

1. Hosted bundle installer

curl -fsSL https://nebuacloud.com/install/nebua-operator.sh | bash -s -- \
  --namespace nebua-system

This downloads the same Nebua agent bundle used by nebua-agent, then runs the bundled nebua-operator-install.sh from inside it. No sudo is required — the script only calls kubectl, it does not touch the host. There is no separate nebua-operator container image to pull: the Deployment runs a stock node:20-alpine, and an initContainer fetches the same public bundle tarball directly inside the cluster and unpacks the compiled operator into a shared volume the main container runs from.

2. Local source installer

If you already have the monorepo checked out:

./nebua-operator-install.sh \
  --namespace nebua-system

3. Manual manifest install

For full control, apply the manifests directly from agents/nebua-operator/manifests:

kubectl apply -f manifests/crds/
kubectl apply -f manifests/rbac.yaml
kubectl apply -f manifests/deployment.yaml

Installer options

--namespace NAME              Namespace for nebua-operator, default nebua-system
--bundle-platform PLATFORM    Bundle platform, default linux-x64
--bundle-version VERSION      Bundle release tag, default latest
--bundle-repo OWNER/REPO      Bundle release repo, default nebua/agent-releases
--bundle-url URL              Exact bundle tarball URL, overrides version/repo/platform
--watch-namespace NAME        Optional namespace scope; omit for cluster-wide watch
--healing-enabled true|false  Enable operator self-healing, default true
--skip-crds                   Do not apply platform.nebua.io CRDs
--no-wait                     Do not wait for deployment rollout
--kubeconfig PATH             Kubeconfig path for kubectl
--context NAME                Kubeconfig context for kubectl

The installer applies the platform.nebua.io CRDs, RBAC, and Deployment, points the Deployment's bundle-fetch initContainer at the requested bundle version/repo/URL and sets the OPERATOR_HEALING_ENABLED environment variable on the main container, and (unless --no-wait) waits for the rollout to finish.

Runtime Configuration

The operator Deployment reads these environment variables:

  • OPERATOR_HEALING_ENABLED: true or false, default true
  • OPERATOR_HEALING_COOLDOWN_MS: minimum time between remediation attempts on the same resource, default 300000 (5 minutes)
  • OPERATOR_HEALING_MAX_BACKOFF_MS: maximum backoff between attempts, default 3600000 (1 hour)
  • OPERATOR_HEALING_MAX_ATTEMPTS: attempts before falling back to a recommendation instead of another action, default 3
  • OPERATOR_HEALING_STUCK_RESOURCE_MS: how long a resource must be stuck before a strategy considers it, default 1200000 (20 minutes)
  • HEALING_EVENT_NAMESPACE: namespace healing Events are emitted into, default nebua-system
  • WATCH_NAMESPACE: set by --watch-namespace; omit to watch the whole cluster

Change these after install with kubectl -n nebua-system set env deployment/nebua-operator KEY=VALUE, then let the rollout restart the pod.

Verify Installation

kubectl -n nebua-system get deployment nebua-operator
kubectl -n nebua-system rollout status deployment/nebua-operator
kubectl -n nebua-system logs deployment/nebua-operator -f

Confirm the CRDs registered:

kubectl get crd | grep platform.nebua.io

You should see nebuapolicies.platform.nebua.io, nebuabackupplans.platform.nebua.io, and nebuaremediations.platform.nebua.io.

Self-Healing Framework

Once running, the operator watches native Kubernetes resources and reconciles them through independent strategies: CrashLoopBackOffStrategy, ImagePullBackOffStrategy, WorkloadReplicaStrategy, NodeHealthStrategy, StorageStrategy, NetworkingStrategy, AutoscalingStrategy, and PolicyStrategy. Each strategy watches, detects, evaluates a safety window, decides whether to act or only recommend, executes a bounded remediation, and reports a Kubernetes Event for the NebuaCloud dashboard to ingest.

Safety state is persisted as annotations on the affected resource (healing.nebua.io/attempts, healing.nebua.io/last-strategy, healing.nebua.io/last-action-at, healing.nebua.io/next-action-at, healing.nebua.io/last-result), so the operator will not loop endlessly — once attempt/cooldown limits are hit it reports a recommendation or a manual-intervention Event instead of retrying.

Custom Resources

The initial API group is platform.nebua.io/v1alpha1:

  • NebuaRemediation: declarative remediation requests such as restartDeployment, cordonNode, uncordonNode, and annotateNamespace
  • NebuaPolicy: policy/compliance intent in audit or enforce mode, enforced natively through ValidatingAdmissionPolicy
  • NebuaBackupPlan: backup orchestration intent for provider-specific backup controllers

Troubleshooting

Operator pod not starting

kubectl -n nebua-system describe deployment nebua-operator
kubectl -n nebua-system get events --sort-by=.lastTimestamp

Common causes:

  • RBAC not applied (manifests/rbac.yaml) — the operator's service account lacks permission to watch/patch resources
  • CRDs not applied — the operator crashes on start if platform.nebua.io CRDs are missing and --skip-crds was used unintentionally
  • bundle-fetch initContainer failing — check kubectl -n nebua-system logs deployment/nebua-operator -c bundle-fetch; usually a bad --bundle-url/--bundle-version/--bundle-repo value or no outbound HTTPS access from the cluster to GitHub

Healing is not happening

  • Confirm OPERATOR_HEALING_ENABLED=true on the Deployment
  • Check whether a resource is inside its OPERATOR_HEALING_COOLDOWN_MS window or has exhausted OPERATOR_HEALING_MAX_ATTEMPTS — look at the resource's healing.nebua.io/* annotations
  • Check kubectl -n nebua-system logs deployment/nebua-operator for the specific strategy's decision

Uninstall

Remove the Deployment and RBAC

kubectl -n nebua-system delete deployment nebua-operator
kubectl delete clusterrolebinding nebua-operator
kubectl delete clusterrole nebua-operator
kubectl -n nebua-system delete serviceaccount nebua-operator

Replace nebua-system with whatever --namespace you installed into. Stopping the Deployment first (before removing CRDs below) avoids the operator logging reconcile errors against resources it can no longer find.

Remove the CRDs

kubectl delete crd nebuapolicies.platform.nebua.io nebuabackupplans.platform.nebua.io nebuaremediations.platform.nebua.io

This deletes every NebuaPolicy, NebuaBackupPlan, and NebuaRemediation object in the cluster along with the CRDs — Kubernetes always garbage-collects custom resources when their CRD is removed. Check what exists first if you're not sure:

kubectl get nebuapolicies,nebuabackupplans,nebuaremediations -A

Skip this step if you plan to reinstall the operator later and want to keep existing policy/backup-plan definitions.

Remove the namespace

Only do this if nebua-system (or your chosen --namespace) was created solely for the operator and holds nothing else:

kubectl delete namespace nebua-system

Uninstalling the operator disables self-healing, policy enforcement, and add-on reconciliation for the cluster. nebua-agent keeps working independently — basic cluster management, k3s bootstrap, and health reporting are unaffected.

Security Considerations

  • The operator uses in-cluster service account credentials only; it does not need outbound internet access to function.
  • Scope --watch-namespace if you want to limit the operator to a subset of the cluster instead of cluster-wide.
  • Review the RBAC manifest (manifests/rbac.yaml) before applying in a shared or regulated cluster — it grants the permissions the healing strategies and policy enforcement need.
  • Healing actions are bounded and reported as Kubernetes Events; review HEALING_EVENT_NAMESPACE events regularly if healing is enabled.

Profile picture

Written with love by Nebuacloud, Private Cloud Infrastructure Automation Platform.