Errors, Retries, and Rate Limits

Handle API errors, retries, and rate limits without surprising your users or automation.

Every error response is a JSON object. The error field is always present and carries a human-readable message. Structured errors — validation failures and canonical business-rule rejections — additionally populate code, message, field, and severity:

{
  "error": "human-readable error message",
  "code": "CLUSTER_DEDICATED_UNSUPPORTED",
  "message": "human-readable error message",
  "field": "type",
  "severity": "error"
}

code is drawn from a fixed set of stable identifiers (for example CLUSTER_DEDICATED_UNSUPPORTED, HA_ENABLE_ON_EXISTING_UNSUPPORTED). When code is present, branch on it rather than pattern-matching the human-readable error/message, which is not a stable contract. The code, message, field, and severity fields are omitted for unstructured errors, so treat them as optional.

Use the HTTP status code together with code (when present) to decide whether to retry, stop, or surface the error to a human.

Status codes

Status	Meaning	Retry?
400	Invalid request, bad input, unknown fields, or validation failure	No, fix the request
401	Missing or invalid authentication	No, refresh credentials
403	Insufficient permissions for the tenant or role	No, use a key with the right role
404	Resource not found	No, fix the path or wait for async creation
409	Resource already exists	No, check the existing resource
412	ETag mismatch because the resource changed after you read it	Yes, re-fetch and retry
429	Rate limited	Yes, wait Retry-After seconds
500	Internal server error	Yes, with backoff, and surface it if it persists
503	Service unavailable, for example while a cluster is still becoming ready	Yes, with backoff

Rate limiting

The API enforces rate limits per IP address. When you exceed the limit you get a 429 response with a Retry-After header.

Retry-After is the number of seconds to wait before retrying.

HEADERS=$(mktemp)
STATUS=$(curl -s -D "$HEADERS" -o /tmp/body -w "%{http_code}" \
  -H "Authorization: Bearer $KUPE_API_KEY" \
  "https://api.kupe.cloud/api/v1/tenants/<tenant>/clusters")

if [ "$STATUS" = "429" ]; then
  WAIT=$(awk -F': ' 'tolower($1)=="retry-after" {gsub(/\r/,""); print $2}' "$HEADERS")
  echo "Rate limited, waiting ${WAIT}s"
  sleep "${WAIT:-5}"
fi

Repeated authentication failures trigger exponential backoff on top of the regular limit. Verify your API key is correct before retrying in a loop.

Optimistic locking conflicts

PATCH requests that include an If-Match header fail with 412 Precondition Failed if the resource changed after you read it. This is recoverable: re-fetch the resource, apply your change against the new state, and retry.

for attempt in 1 2 3; do
  ETAG=$(curl -sD - -o /dev/null \
    -H "Authorization: Bearer $KUPE_API_KEY" \
    "https://api.kupe.cloud/api/v1/tenants/<tenant>/clusters/<cluster>" \
    | awk -F': ' '/^[Ee][Tt][Aa][Gg]:/ {gsub(/\r/,""); print $2}')

  STATUS=$(curl -s -o /tmp/body -w "%{http_code}" -X PATCH \
    -H "Authorization: Bearer $KUPE_API_KEY" \
    -H "Content-Type: application/json" \
    -H "If-Match: $ETAG" \
    "https://api.kupe.cloud/api/v1/tenants/<tenant>/clusters/<cluster>" \
    -d '{"version": "1.32"}')

  if [ "$STATUS" = "200" ]; then
    echo "Updated"
    exit 0
  elif [ "$STATUS" = "412" ]; then
    echo "Conflict, retrying (attempt $attempt)"
    continue
  else
    echo "Failed with $STATUS" >&2
    cat /tmp/body >&2
    exit 1
  fi
done

echo "Gave up after 3 attempts" >&2
exit 1

Retry strategy

A safe default for retryable status codes 429, 500, and 503:

• exponential backoff with jitter, starting at 1 second and capping at 30 seconds
• up to 5 attempts
• retry GET, DELETE, and PATCH when you use If-Match
• treat POST more carefully, because success may already have happened before the retry

If you hit 429 repeatedly, reduce request concurrency rather than only waiting longer.