KAIF·k8s agentic ai fabric
Pillar 02 · run

Deployment Lifecycle

The deployment lifecycle takes a signed agent artifact and gets it safely serving traffic. It is the Kubernetes lifecycle for agents: operators reconcile desired state; packages define the shape; GitOps drives rollout; KEDA and Kueue scale workers and GPUs; an AI gateway routes traffic; admission and runtime policies enforce security; and guardrails decide what an agent is actually allowed to do.

Modules involved
plan hub →
kaif-platformThe technology substrate every other module runs on.kaif-deployGet an agent safely serving traffic.kaif-evalOne evaluation spine, four surfaces — design, build, deploy, runtime.kaif-guardAll guardrails and policies. Publishes bundles kaif-deploy enforces.kaif-identityAuthN and AuthZ across users, agents, tools and tenants.
What good looks like

Outcomes

  • Every change reaches production through Git — no kubectl-driven hotfixes.
  • Operators reconcile agents, models and prompts as first-class Kubernetes objects.
  • Models scale to zero when idle and burst onto GPUs under load.
  • New agent versions ship as canaries or shadows before taking real traffic.
  • An agent that misbehaves is contained — by policy, by sandbox and by network.
  • Tenants share one cluster but never share each other's data, tokens or GPUs.
The loop

How it actually runs

  1. step 01
    Promote in Git
  2. step 02
    Operator reconciles
  3. step 03
    Rollout canary / shadow
  4. step 04
    Autoscale on demand
  5. step 05
    Route via AI Gateway
  6. step 06
    Enforce policy & observe
Cloud-native enablers

Technology on Kubernetes

Every capability maps to a CNCF or Kubernetes-native project. No parallel stack — the same cluster runs your agents and your platform.

Kubernetes lifecycle · operators & CRDs

  • Agent / Prompt / Model CRDsDeclare agents and their dependencies as K8s objects.
  • KServe controllerReconciles InferenceService for every model.
  • Argo CD ApplicationSetGenerate one app per agent across clusters.
  • OperatorHub / OLMLifecycle-managed install of platform operators.
  • Kubebuilder · Operator SDKBuild the KAIF operators themselves.

Packaging

  • HelmTemplated charts per agent type.
  • KustomizePer-env overlays without forking.
  • OCI Helm + KitOpsDistribute charts and ModelKits through the registry.
  • Carvel kapp / yttDeclarative app bundles with diff and prune.

GitOps · rollout

  • Argo CD / FluxDeclarative, Git-driven rollout to every cluster.
  • Argo RolloutsCanary, blue-green and shadow deployments with analysis.
  • FlaggerProgressive delivery driven by metrics.
  • KargoStage-to-stage promotion across envs.

Scale · runtime

  • KServe + vLLMScalable LLM inference with continuous batching.
  • KnativeScale agent workers to zero and back on request.
  • KEDAEvent-driven scaling on queue depth, tokens, custom metrics.
  • KueueFair-share GPU scheduling for batch agent jobs.
  • NVIDIA GPU Operator + KarpenterRight-size and burst GPU nodes.
  • HPA · VPAClassic pod-level auto-sizing.

Routing · traffic

  • Envoy AI GatewayModel routing, fallback, token & cost budgets.
  • Istio / Gateway APIMesh-level routing, mTLS, mirroring.
  • LiteLLM proxyProvider-agnostic LLM routing surface.
  • Cilium L7 policiesEgress control per-tool, per-endpoint.

Security · policies · guardrails

  • Kyverno · OPA GatekeeperAdmission: only signed images, approved models.
  • OPA (per-request)Authorize tool calls and data access.
  • gVisor / Kata ContainersSandbox tool execution from the host kernel.
  • NetworkPolicies + CiliumLimit which APIs an agent can reach.
  • SPIFFE / SPIREWorkload identity for agents and tools.
  • NeMo Guardrails · Llama GuardRuntime input/output safety rails.
  • Falco · TetragonRuntime detection of suspicious agent behavior.

Multi-tenancy

  • vClusterVirtual clusters per team or tenant.
  • CapsuleNamespace-based tenancy with quotas.
  • ResourceQuotas · LimitRangesCap GPU, tokens and storage per tenant.
  • External SecretsPer-tenant credentials without sharing.
Cross-cut

How the Harness plugs in

The Harness gates every deployment: an agent is promoted only if it has a signed image, an approved model, current eval results above threshold, and an attached policy bundle. At runtime the Harness publishes routing rules, rate limits and guardrails as config that the AI Gateway, OPA and the safety sidecars enforce — so deployment-time governance and runtime behavior never disagree.
← DevelopmentEvaluation Lifecycle →