High Availability

High Availability (HA) gives a Kupe cluster a 3-replica control plane backed by a chart-managed external etcd StatefulSet with hard anti-affinity, encryption at rest, and a PodDisruptionBudget that protects quorum. With HA enabled the cluster API stays up through node drains, kured reboots, autoscaler scale-downs, and single-host failures.

HA is off by default and opt-in per cluster. Tenants can run a mix of HA and non-HA clusters under the same tenant pool.

When HA is the right choice

HA is for clusters where the cluster API itself needs to stay up during platform maintenance. Typical cases:

Production clusters that GitOps controllers, deploy pipelines, or kubectl automation talk to continuously.
Multi-tenant internal platforms where many teams’ deploys would otherwise stack up during a 1–3 minute API blip.
Compliance contexts that require demonstrable redundancy of the control plane.

HA is not needed for:

Dev / preview clusters where a brief API blip is acceptable.
Workload availability, which remains the tenant’s responsibility via Deployments, replicas, and PodDisruptionBudgets on their own workloads. HA protects the control plane only.

Cost

HA adds a flat hourly rate per cluster, billed by the hour from the moment 3/3 replicas are ready, not from when the cluster is created.

Property	Value
Rate	£0.0205/hour (≈ £15/month at 730 hours)
Billed from	The first time the operator confirms 3/3 replicas ready (`status.haEnabledAt`)
Billed until	Cluster deletion (`metadata.deletionTimestamp`)
Pool impact	HA replica pods do not count against your tenant pool; they are platform-managed

The HA charge replaces what would otherwise be billed at per-resource rates for the extra 2 control-plane replicas plus the deployed-etcd StatefulSet.

A two-day spin-up reads as ~£0.99 on the invoice rather than £15, because billing is by the hour rather than by the month. Clusters deleted before the next billing run are still billed for their unbilled HA hours: the operator stamps a pending charge before removing the cluster.

Enable HA at cluster creation

Open Clusters → Create Cluster.
Fill in name, version, resources.
Under Reliability, check High Availability.
Review the cost preview. The monthly total includes the HA charge.
Click Create.

The cluster card and details page will show an HA chip and an On (3 of 3 replicas ready) row once the operator confirms HA. Billing starts at that point.

kupe cluster create prod-eu1 \
  --version 1.35 \
  --cpu-limit 8 --memory-limit 32Gi --storage-limit 200Gi \
  --high-availability \
  --wait

kupe cluster list shows the HA state in the HA column. kupe cluster get <name> shows the full HA detail line including the billing anchor timestamp.

resource "kupe_cluster" "prod_eu1" {
  name              = "prod-eu1"
  display_name      = "Production EU1"
  version           = "1.35"
  high_availability = true

  resources = {
    cpu     = "8"
    memory  = "32Gi"
    storage = "200Gi"
  }
}

output "prod_eu1_ha_phase" {
  value = kupe_cluster.prod_eu1.ha_phase   # ha-healthy | ha-degraded | ha-unavailable
}

The provider exposes ha_phase, ha_configured, ha_enabled_at, ha_replicas_ready, ha_replicas_desired, ha_etcd_replicas_ready, and ha_etcd_replicas_desired as computed attributes so downstream automation can branch on operational state without inspecting conditions. The etcd replica counts are surfaced separately because the deployed-etcd path runs etcd in its own StatefulSet, and etcd quorum loss blocks writes even when the apiserver replicas are healthy.

high_availability carries RequiresReplace: changing it on an existing resource causes Terraform to destroy and recreate the cluster. Use a blue-green swap workflow instead (see below).

POST /api/v1/tenants/{tenant}/clusters
Content-Type: application/json

{
  "name": "prod-eu1",
  "version": "1.35",
  "highAvailability": true,
  "resources": { "cpu": "8", "memory": "32Gi", "storage": "200Gi" }
}

GET /api/v1/tenants/{tenant}/clusters/{name} surfaces haPhase, haConfigured, haEnabledAt, haReplicasReady, haReplicasDesired, haEtcdReplicasReady, and haEtcdReplicasDesired under status. The etcd counts are reported separately from the apiserver counts so callers can detect etcd quorum loss (which blocks writes) even when the apiserver replicas read healthy.

PATCH /api/v1/tenants/{tenant}/clusters/{name} rejects HA toggles in both directions:

false → true: HA_ENABLE_ON_EXISTING_UNSUPPORTED
true → false: HA_DISABLE_UNSUPPORTED

Switching an existing cluster to HA

HA is set at creation and cannot be toggled later. The standard pattern is a blue-green cluster swap managed through your existing GitOps or deployment tooling:

Create a new HA cluster with the same shape (version, resources) as your existing cluster:

kupe cluster create prod-eu1-ha \
  --version 1.35 \
  --cpu-limit 8 --memory-limit 32Gi --storage-limit 200Gi \
  --high-availability

Deploy your workloads to the new cluster by pointing your Argo CD ApplicationSet / Flux GitRepository / kubectl apply workflow at the new cluster’s kubeconfig. Existing GitOps manifests should work unchanged.
Verify the new cluster: run smoke tests, check metrics, and confirm integrations work.
Swap traffic. Update DNS, ingress, or load-balancer config to point at the new cluster’s endpoint.
Delete the old cluster once you’ve verified the swap:
Terminal window
```
kupe cluster delete prod-eu1
```

This is the same shape as control-plane upgrades on EKS, GKE, and AKS: you create a new cluster and migrate to it rather than reshaping an existing control plane.

States

`haPhase` value	Meaning
`pending`	HA requested, control plane and/or etcd not yet 3/3 ready
`ha-healthy`	3/3 replicas ready, full redundancy
`ha-degraded`	2/3 replicas ready, quorum intact, API serving; one replica failure away from outage
`ha-unavailable`	≤1/3 replicas ready, quorum lost, API not serving

Terraform exposes these raw values on ha_phase. The console and CLI collapse the bottom two into a single Degraded label, distinguished by colour rather than wording, so a cluster showing “Degraded” in the UI may be either quorum-intact (ha-degraded) or quorum-lost (ha-unavailable). Check ha_phase directly when the difference matters.

For most tenants the cluster sits in ha-healthy indefinitely; ha-degraded typically clears within minutes as the operator reschedules the failed replica. ha-unavailable indicates a real incident; file a support ticket.

What HA does not protect against

HA makes the control plane resilient to single-replica and single-host failures. It does not cover:

Cross-region outages. HA is single-region; all three replicas live in the same host cluster.
Workload availability. Your pods, services, and storage remain your responsibility. HA does not add replicas to your workloads or restart failed pods faster.
Platform-wide admission outages. If the platform’s admission webhooks themselves go down, control-plane replication does not help. Kupe hardens those separately.
Application-level data loss. etcd holds the cluster’s Kubernetes state, not your application data. Application backups remain your responsibility.

Frequently asked questions

Can I enable HA on some clusters and not others?

Yes, HA is per-cluster. A typical pattern is HA on prod and single-replica on dev and staging.

Does HA increase my workload capacity?

No. The 3 HA replicas are the control plane (API server, controller manager, scheduler, etcd). Your workload pods run in their own namespaces against this control plane; HA doesn’t change the resources available to them.

Is etcd encrypted at rest?

Yes. Every cluster (HA or not) encrypts Secrets at rest in etcd with a per-cluster AES-CBC key stored in the platform’s vault. HA clusters inherit the same protection.

Can I see when HA started for billing reconciliation?

Yes. Console shows the haEnabledAt timestamp in the cluster details. CLI: kupe cluster get <name> includes it. Terraform: read the ha_enabled_at computed attribute. The same timestamp drives the invoice line.

What happens to the HA charge if I delete the cluster mid-month?

The operator stamps the unbilled HA hours into your tenant status before removing the cluster. The next billing run picks them up and adds them to the invoice. You won’t be over-charged or under-charged.