Troubleshooting

Each section starts from the message you see. Clients print the registry’s message with their own prefix, for example UNAUTHORIZED: from crane or unauthorized: from Docker.

unauthorized when signing in

docker login, or the first push or pull, fails with unauthorized. The registry does not say why a sign-in failed. Check, in order:

• The session expired. A login session lasts 12 hours. Run kupe registry login again.
• The username is incomplete. An access token’s username is the whole kupe+<tenant>+tok-… value shown with the token, not the token’s name or ID.
• The token has expired or was revoked. kupe registry token list shows what exists and when each token expires. A token is also revoked when the member who created it leaves the tenant.
• The secret was copied incompletely. It is 49 characters long and starts with kupecr_. Copy it again from where you stored it, or create a new token: the secret cannot be shown again.

unauthorized to access repository: …, action: push

The credential signed in, but it cannot push. The registry names the repository and the action it refused:

unauthorized to access repository: acme/api, action: push

• A pull-only credential. A pull access token, or the login session of a readonly member, can pull but not push. Use a push token, created by an admin.
• A demoted member. When an admin becomes a readonly member, the push tokens and login sessions they created are revoked. Create a new push token as an admin.
• The wrong tenant in the path. A credential reaches its own tenant’s registry only. Check that the image path starts with registry.kupe.cloud/<your tenant>/.
• The tenant is suspended. While the tenant is suspended, pushes and pulls are refused; see Suspension.

unauthorized to access repository: …, action: pull

The same causes as for a push, for pulls: the wrong tenant in the path, a token that has expired or was revoked, or a suspended tenant.

Push refused at the storage limit

DENIED: adding 10.0 MiB of storage resource, which when updated to current usage of 95.0 MiB will exceed the configured upper limit of 100.0 MiB.

The push would take the registry past its storage limit. Delete images you no longer need, including untagged ones, or raise the limit with kupe registry update --storage-limit <size> up to your plan’s cap. Pulls keep working while the registry is full. See Managing images: the layers a refused push uploaded count until the weekly clean-up.

kupe registry login refused with session_requires_user

Error: login sessions are for people signed in with kupe; use an access token for automation
  sign in with:  kupe auth login
  for automation, use an access token:  kupe registry token create --name NAME --access pull|push

The CLI is using an API key, from --token, KUPE_API_TOKEN or the configuration. A login session belongs to a person: sign in with kupe auth login, or create an access token for the automation.

Registry delete refused with deletion_protected

Error: registry "registry" has deletionProtection enabled: set it to false before deleting
  turn deletion protection off first:  kupe registry update --deletion-protection=false

Deletion protection is on. Turn it off, then delete. Deleting the registry deletes every image in it; see Deleting the registry.

ImagePullBackOff in a connected cluster

The pod’s events show ErrImagePull, then ImagePullBackOff. Check:

• The pod is older than the connection. Pods get the pull credential when they are created. Restart the workload with kubectl rollout restart deployment/<name>.
• The connection is not ready. kupe registry connection get <cluster> shows the phase and message. A ready connection says connected: new pods in cluster <cluster> can pull from registry.kupe.cloud/<tenant>; restart workloads started earlier to use it. The RegistryConnectionNotReady alert fires after 15 minutes of a connection not being ready.
• The cluster is not connected. Connect it; see Connect a cluster.
• The image path is wrong. It must be registry.kupe.cloud/<tenant>/<repository>:<tag>, with your tenant name. kupe registry image get <repository>:<tag> prints the exact pull references.
• The tag was deleted or moved. kupe registry image list <repository> shows the tags that exist now. Pin deployments by digest to stop a moved tag changing what they run.
• The image has no build for the node. Kupe’s nodes are linux/amd64. An image built only for linux/arm64 cannot run there; build it with --platform linux/amd64.
• The tenant is suspended. Running pods keep running, but a pod that needs to pull cannot start until the tenant is reactivated.

Push Secret missing from a namespace

kupe registry connection get <cluster> lists each push namespace with its phase and message.

• The namespace is not a push namespace. Add it with kupe registry connection update <cluster> --add-push-namespace <namespace>.
• The namespace does not exist yet. The Secret appears once you create the namespace in the cluster. Until then the connection’s message reads the push credential is not yet delivered in namespace "<namespace>"; retrying.
• A Secret of that name already exists. The connection is Failed and the namespace’s message names the clash, for example Secret kupe-registry-push already exists in namespace "ci", not created by the platform; remove it, or connect with another secretName. Delete or rename your Secret, and the platform delivers its own. Pulls keep working in the meantime.
• The registry has 20 push namespaces. A connection with a namespace over the registry’s limit says so in its message. Remove a push namespace from another connection.

Related pages

• Registry
• Push and pull
• Connect a cluster
• Access tokens and CI
• Support