Upgrading

Every upgrade follows the routine steps. Some releases also need release-specific steps, listed newest first: read every entry between your current version and the target before you start.

Routine upgrade

Read the release notes for every version between your current version and the target first. Confirm the target chart is available before changing the Application:

helm show chart oci://quay.io/nebari/charts/nebari-llm-serving --version <version>

A GitHub Release can appear before its chart is published. Pointing ArgoCD at an unavailable version reports Sync: Unknown with a ComparisonError, even while health still shows Healthy. Wait for chart publication before proceeding; see the release publication caveat in the README.

Bump targetRevision in the pack’s ArgoCD Application (Installation section 8.1) to the new chart version, push, and re-apply the Application with kubectl apply -f (or let your app-of-apps sync it). Pushing alone does not update an Application that was installed manually. With automated sync on, ArgoCD then rolls it out; otherwise sync it by hand.

CRDs. The chart ships the LLMModel and PassthroughModel CRDs in its crds/ directory. ArgoCD applies them on every sync, so they move with the operator as long as the Application does not set skipCrds: true. Plain helm upgrade installs crds/ only on first install and never updates it, so a Helm-managed install must apply the CRDs from the target release before upgrading:

helm pull oci://quay.io/nebari/charts/nebari-llm-serving --version <version> --untar
kubectl apply --server-side --force-conflicts -f nebari-llm-serving/crds/
kubectl wait --for=condition=Established --timeout=60s \
  crd/llmmodels.llm.nebari.dev crd/passthroughmodels.llm.nebari.dev
helm upgrade nebari-llm-serving oci://quay.io/nebari/charts/nebari-llm-serving \
  --version <version> -n nebari-llm-serving-system -f <your-values>.yaml

For a plain Helm install, --force-conflicts transfers ownership of the CRD schema fields from Helm to kubectl: Helm owns them from the initial install but does not update them during upgrades. If a separate tool manages the CRDs (such as a GitOps repository or platform CRD pipeline), update them through that tool instead of forcing ownership away from it.

Stop if the CRD apply fails. An old CRD can still report Established, and helm upgrade can succeed while leaving its schema unchanged.

If the CRDs lag behind the chart, the API server keeps validating against the old schema: fields the release added are rejected or dropped, so those features do not work until the CRDs are updated.

Never delete a CRD to upgrade it

Deleting a CRD deletes every resource of that kind. Removing the LLMModel or PassthroughModel CRD removes every model, and their cleanup deletes the API keys users minted for them.

Then re-run the checks in Installation section 8.2.

Supported gateway versions

The supported deployment target is Envoy AI Gateway 1.1.0, Envoy Gateway 1.8.1, and Gateway API 1.5.1. Use the released upstream Helm charts, including the separate AI Gateway CRD chart. The pack does not install those controllers.

Release-specific steps

Newest first. A release with no entry needs only the routine steps.

Upgrading from 0.1.4 or earlier: Envoy AI Gateway 1.1

Releases after 0.1.4 emit aigateway.envoyproxy.io/v1beta1 resources and target the gateway versions above. An installation on 0.1.4 or earlier needs these steps.

Existing provider settings

The operator translates the legacy spec.provider.schemaVersion setting into Envoy’s schema.prefix: api/v1 still reaches /api/v1/chat/completions, and the default remains /v1. Bedrock uses its own native paths without a prefix. API schema versions and URL prefixes remain separate in the provider adapter.

Both served and external-provider routes use aigateway.envoyproxy.io/v1beta1. Each endpoint declares its hostname, so /v1/models includes its own catalog without depending on the other endpoint being enabled. Listener references and per-model authentication remain in place. Hostname scoping is not user-level catalog filtering; model invocation still enforces API-key scope and JWT groups. Disabling a provider endpoint removes its route and catalog entries. Its auth policy stays until the generated HTTPRoute is gone, and issued keys are preserved for re-enabling the endpoint.

If the gateway is already on a supported release and only the pack is changing, roll the AI Gateway controller after installing the new operator. This lets it restore cleanup finalizers that older pack operators removed from its resources.

Provider hostnames are normalized to lowercase and have a single trailing DNS root dot removed before gateway resources are generated. The stored spec is not rewritten. Review existing PassthroughModel.spec.provider.hostname values before upgrading: underscores, empty labels, URLs, ports, paths, and whitespace are rejected. Older webhooks admitted some of these values; correct any remaining invalid names before upgrading to avoid reconciliation failures.

Upgrade an installation on AI Gateway 0.5

Follow the upstream controller upgrade policy and compatibility matrix. Use staged revisions, rather than changing every controller independently:

1. Record the current chart and operator revisions and inventory all gateway resources, including non-LLM routes on a shared gateway. Check the intervening Envoy Gateway release notes for policy changes. Keep CRDs during rollback; deleting them also deletes their resources.
2. Install the compatible Gateway API and Envoy Gateway CRDs and upgrade Envoy Gateway to 1.8.1 using the official chart. CRD updates must be explicit: a Helm controller upgrade alone does not reliably update installed CRDs.
3. Before crossing AI Gateway 0.6, migrate any external OpenAI backends from schema.version to schema.prefix. If the old operator still reconciles these resources, pause that reconciler during the handover. Move any custom AIGatewayRoute.spec.filterConfig into GatewayConfig; this pack does not generate filterConfig.
4. Upgrade AI Gateway CRDs and controller to 0.7, then update the pack’s CRDs as in the routine steps and deploy the new pack operator. Until the CRDs are updated, the old PassthroughModel schema still requires a hostname and an API-key Secret and rejects the new backend and credential fields, so wait for the new operator and webhook before applying Bedrock resources. Version 0.7 supports hostname-scoped catalogs and the v1beta1 resources the operator now emits. Confirm that all resources reconcile before proceeding to AI Gateway 1.1, using 1.0 as an intermediate controller release. AI Gateway 1.0 has a namespace-discovery bug when the Gateway lives in the Envoy Gateway controller’s namespace: it cannot refresh the sidecar config. In that layout, retain the previous data-plane pod and proceed to 1.1, which deduplicates the namespace lookup.
5. Roll the Envoy data plane after each AI Gateway controller upgrade so new pods receive its matching ext-proc sidecar. Use an EnvoyProxy pod-template annotation to request the rollout; do not edit generated Deployments. Preserve the ServiceAccount and cloud workload identity. With Argo CD, retain the cert-manager webhook certificate configuration.
6. Check both inference endpoints, model catalogs, streaming, rejected credentials, local InferencePool models, and non-LLM routes. Expect active connections to reconnect during a data-plane rollout. Only finish the upgrade when the desired GitOps revisions and actual pod images agree.

The v1alpha1 AI Gateway API remains served upstream, but new writes use v1beta1. Reapplying resources migrates their stored representation; do not remove old CRD storage versions until every stored object has been migrated.

For rollback, restore compatible controller/operator revisions through GitOps while retaining the installed CRDs. Returning to AI Gateway 0.5 also requires restoring its path-prefix and catalog configuration; an image-only rollback is not sufficient.

Scope of the upgrade

AI Gateway 1.1 makes newer protocol translation, token counting, streaming controls, and tracing available. It does not automatically enable additional providers, failover, or tracing, and does not grant cloud model access. Provider adapters still select the appropriate upstream schema and authentication.

Check generated resources against upstream CRDs

The operator’s TestGatewayAdmission uses a local Kubernetes API server with the released AI Gateway CRDs. It checks admission and stored fields, including path prefixes and hostname scoping. No cluster or cloud credentials are used.

Render ai-gateway-crds-helm version v1.1.0 into a temporary directory, set AI_GATEWAY_CRD_DIR to that directory and KUBEBUILDER_ASSETS to the binaries installed by make setup-envtest, then run from operator:

go test ./internal/controller/reconcilers -run TestGatewayAdmission -v