Cluster Policies

Understand the workload policies Kupe enforces before a deployment reaches the cluster.

Policies that affect your workloads

Policy	Mode	What it means for you
No privileged or host-level access	Enforce	Privileged containers, host networking, host PID/IPC, and hostPath volumes are rejected
Non-root containers	Enforce	runAsNonRoot: true and allowPrivilegeEscalation: false are required
Drop all capabilities	Enforce	capabilities.drop: ["ALL"] is required; only NET_BIND_SERVICE may be added back
Resource requests required	Enforce	CPU and memory requests are required; defaults are applied if you omit them
Default service account token mount	Automatic	The default ServiceAccount does not mount a token unless you opt in
No NodePort / LoadBalancer services	Enforce	Use ClusterIP plus an HTTPRoute for external access
HTTPRoute hostname validation	Enforce	Public route hostnames must stay within your tenant’s allowed domain space
Seccomp profile	Audit	RuntimeDefault or Localhost is recommended but not yet required
Read-only root filesystem	Audit	readOnlyRootFilesystem: true is recommended but not yet required
Image registry policy	Audit	Registry usage is logged for review; deploys are not blocked

Default resource requests and limits

If your pod omits resources.requests or resources.limits, Kupe applies sensible defaults before policy validation runs. Requests come from the operator-managed namespace defaults, and omitted limits are then copied from requests by policy. That means a simple workload is not rejected just because you forgot to set resource fields.

The defaults are:

Field	Default value
resources.requests.cpu	100m
resources.requests.memory	128Mi
resources.limits.cpu	Same as requests (auto-set)
resources.limits.memory	Same as requests (auto-set)

How limits are set: if you omit limits, the platform automatically sets them equal to your requests — giving your pod Guaranteed QoS (the highest priority class in Kubernetes). This means your pod won’t burst above its request, but it also won’t be evicted before Burstable pods during node pressure.

If you want burst headroom (e.g., a web server that idles at 100m but spikes to 400m during request peaks), set both requests and limits explicitly:

resources:
  requests: { cpu: 100m, memory: 128Mi }
  limits: { cpu: 400m, memory: 512Mi }

As a practical default, keep per-pod limits close to 4× requests for CPU and 2× requests for memory unless the workload has a clear reason to burst higher. Kupe does not currently reject higher ratios in tenant namespaces because some mirrored system pods need larger ratios; ratio enforcement may move into a tenant-scoped policy later with advance notice.

These defaults only apply to containers that don’t set their own resources. If you declare requests, that’s what runs — limits default to match your requests. If you declare both, your values are used as-is.

Billing is on actual usage, not requests. You pay for the CPU and memory your workloads actually consume, not what they reserve. Setting higher requests reserves more scheduling capacity (and counts against your cluster’s quota) but doesn’t increase your bill unless your workload actually uses it. See Cluster resources and quotas for how quota and billing interact.

Reserved namespaces

Some namespaces inside your cluster are reserved for Kupe platform infrastructure and are not for general workload use:

Namespace	Purpose
kube-system	Reserved for cluster components Kupe manages on your behalf — DNS, metrics, in-cluster services.

You can read resources in kube-system (for diagnostics) but should not treat it as a place to deploy your own workloads:

• Custom workloads deployed into kube-system will not run. The platform rejects them at admission. Your kubectl apply may succeed against your cluster’s apiserver, but the pod will not be scheduled and you’ll see an admission error when the platform tries to materialise it.
• Do not modify, delete, or scale the Kupe-managed components in kube-system (CoreDNS, kube-state-metrics, etc.). Doing so will break cluster functionality (DNS resolution, autoscaling signals) for your own workloads — you’d be cutting off the branch you’re sitting on, and Kupe support cannot diagnose or restore your cluster while it’s in that state.
• Deploy your workloads to any other namespace. Create whatever namespaces fit your application’s structure (e.g. production, staging, monitoring, tools). The full cluster is yours; only kube-system is off-limits.

This is the same model as the major managed-Kubernetes providers (EKS, GKE, AKS) — kube-system is part of the managed control plane, not your application surface.

Platform exclusions

A few platform-managed resources are deliberately excluded from your tenant quota and reserved for platform infrastructure — most notably the platform StorageClass (kupe-platform-storage).

• PersistentVolumeClaims cannot use kupe-platform-storage. That StorageClass is excluded from your tenant’s storage quota and is reserved for platform components. Use the default StorageClass for your own volumes — your tenant quota covers it.

If a deploy is rejected for referencing a platform-reserved resource, switch to the tenant-facing equivalent (for storage, the default StorageClass) or contact support if you believe you need access.

Capability exemptions

Every tenant container must drop all Linux capabilities. This is mandatory — there is no self-service way to opt out. The only capability you may add back is NET_BIND_SERVICE (for binding ports below 1024):

containers:
  - name: app
    securityContext:
      capabilities:
        drop: ["ALL"]
        # Optional, allowlist only — for binding ports below 1024.
        add: ["NET_BIND_SERVICE"]

Any other capability in add — or a missing drop: ["ALL"] — is rejected at admission.

Caution

Earlier versions of the platform accepted a kupe.cloud/capability-exemption pod annotation that unlocked additional capabilities. That mechanism has been removed as a security fix. The annotation is ignored; workloads relying on it are now rejected and should switch to the process below.

Some workloads legitimately need a capability beyond the allowlist — for example Redis using mlock (IPC_LOCK), profilers and debuggers (SYS_PTRACE), or VPN sidecars (NET_ADMIN, NET_RAW). For these, contact support: exemptions beyond NET_BIND_SERVICE are granted through a platform-issued policy exception created by a Kupe operator, scoped to the specific workload that needs it.

Important rules:

1. drop: ["ALL"] is always required. No annotation or setting on your side disables the check.
2. Only NET_BIND_SERVICE may be added back without a platform-issued exception.
3. Request exceptions sparingly. Most workloads do not need extra capabilities.
4. Other policies still apply. A capability exception only affects the Drop all capabilities rule. Enforced controls such as runAsNonRoot and allowPrivilegeEscalation still reject violating workloads, while audit-only checks remain informational.

If you’re not sure whether you need an exception, try without it first — most modern images work cleanly with drop: ["ALL"], and ports below 1024 only need the NET_BIND_SERVICE add-back.

Fixing common violations

When a policy blocks your deployment, the error message Kupe returns includes the exact YAML to add. This section collects the most common ones for reference.

”Tenant containers must set runAsNonRoot: true”

Add a securityContext block to your pod spec:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 65532 # nobody
        runAsGroup: 65532
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: app
          image: ghcr.io/your-org/app:v1.0.0
          securityContext:
            runAsNonRoot: true
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]

If your image only runs as root, the fastest fix is usually to switch to a non-root base image (Chainguard, Red Hat UBI, distroless, or Bitnami’s nonroot variants). Most official images on Docker Hub (postgres, redis, nginx, mysql, mariadb) support non-root out of the box.

”Containers must drop ALL capabilities”

Two options:

Option 1 — drop everything (the happy path):

containers:
  - name: app
    securityContext:
      capabilities:
        drop: ["ALL"]

Option 2 — drop all and add NET_BIND_SERVICE back (the only allowed add-back):

containers:
  - name: app
    securityContext:
      capabilities:
        drop: ["ALL"]
        add: ["NET_BIND_SERVICE"]

Anything beyond NET_BIND_SERVICE requires a platform-issued policy exception — see Capability exemptions above.

”CPU and memory requests are required”

You rarely need to set these yourself — Kupe applies sensible defaults for any container that omits them. If you’re hitting this error anyway, fill in the missing request field:

containers:
  - name: app
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      # limits are optional; omitted limits default to match requests.

“NodePort and LoadBalancer services are not allowed”

Change your Service to ClusterIP and create an HTTPRoute for external access:

apiVersion: v1
kind: Service
metadata:
  name: my-app
spec:
  type: ClusterIP # ← was NodePort or LoadBalancer
  selector:
    app: my-app
  ports:
    - port: 80
      targetPort: 8080

Then follow the standard route pattern in HTTP Routes to publish the service.

If you see a registry policy warning or error

On the standard Kupe tenant policy set, the registry policy is currently in audit mode. Registry usage is logged, but deploys are not blocked. If your cluster is using a stricter custom policy and a deploy is blocked, contact support.

Audit mode vs enforce mode

Some policies run in audit mode — violations are logged to the platform but don’t block the deploy. This gives you visibility before a rule becomes a hard requirement.

Currently in audit mode:

• Seccomp profile requirement — most modern runtimes set RuntimeDefault automatically; audit mode helps catch workloads that still need adjustment.
• Read-only root filesystem — breaks many common images that write to /tmp or /var/log; audit lets you migrate at your own pace.
• Image registry policy — currently informational; deploys continue while registry usage is recorded for visibility.

These may be promoted to enforce mode in a future update with advance notice.

Related pages

• Container Security
• Network Isolation
• HTTP Routes
• Cluster limits and quotas