Notifications

Notifications control how alerts reach your team. Once an alert is firing, Alertmanager routes it to one or more receivers based on the routing rules you configure in the console.

Supported receiver types

Kupe Cloud supports five receiver types in the console today:

Type	Required fields	Notes
Slack	Webhook URL	Supports custom title, text, and color templates
Microsoft Teams	Webhook URL	Supports custom title and text templates
PagerDuty	Routing key or legacy service key	Severity is derived from the alert labels
Email	To, From, SMTP server	SMTP auth is optional. TLS is recommended
Webhook	URL	Sends the standard Alertmanager webhook payload

All receiver types support Send resolved notifications, which is enabled by default.

Routing model

Configure routes in Alerting > Notifications > Routing.

Default route

The default route catches alerts that do not match a more specific rule. Set a default receiver here before you add more granular routes.

Matchers

Routes match on alert labels. The console supports exact match, negative match, regex match, and negative regex match.

Common labels include:

• severity
• alertname
• cluster
• namespace
• team

Route order

Routes are evaluated from top to bottom. The first matching route wins unless Continue matching is enabled.

Use Continue matching when you want the same alert to reach more than one receiver, such as Slack and PagerDuty.

Grouping and timing

Each route can override how alerts are grouped and re-sent.

Setting	Default	Purpose
Group by	alertname	Batches related alerts together
Group wait	30s	Delay before the first notification for a new group
Group interval	5m	Minimum wait before updates to an existing group
Repeat interval	4h	How often to repeat an unchanged firing alert

Lower values make notifications faster and noisier. Higher values reduce noise but slow down delivery.

Templates

Slack, Teams, and PagerDuty receivers include useful default templates, so you do not need to write templating to get started. Email and Webhook receivers use the standard Alertmanager payloads.

You can override template fields with Alertmanager Go templates when you need custom formatting. Common fields include:

• {{ .Status }}
• {{ .Alerts }}
• {{ .CommonLabels }}
• {{ .CommonAnnotations }}

Example custom Slack text:

{{ range .Alerts -}}
*{{ .Labels.alertname }}* ({{ .Labels.severity }})
{{ .Annotations.summary }}
Namespace: {{ .Labels.namespace }}
{{ end }}

Test a receiver

Each receiver has a Test action in the console.

When you run a test:

1. the console creates a short-lived synthetic alert
2. Alertmanager routes it through the normal routing tree
3. the alert auto-resolves after a few seconds

If the test goes to the wrong place, review your route order and matchers.

Troubleshooting

Test succeeds but nothing arrives

• verify the webhook URL, SMTP details, or PagerDuty key
• check for trailing spaces or a revoked webhook
• confirm the route actually points at the receiver you tested

Notifications go to the wrong receiver

• reorder routes so more specific rules are above broader ones
• check whether Continue matching is enabled on an earlier route
• review the labels on the alert that is actually firing

Notifications look too plain

Edit the receiver and review the template fields. Slack, Teams, and PagerDuty support richer formatting than email or generic webhooks.

Manage notifications as code

Everything you can configure in Alerting > Notifications in the console is also available as a REST API and as Terraform resources, so you can manage notification configuration in the same repository as the rest of your infrastructure.

Resources

The Kupe Terraform provider exposes three resources for the Alertmanager configuration:

• kupe_alertmanager_receiver — one resource per named receiver (Slack, PagerDuty, email, webhook, Teams, …). The receiver body is authored as JSON via jsonencode() so any receiver type Alertmanager supports works without a provider release.
• kupe_alertmanager_routes — singleton-per-tenant resource that owns the entire ordered list of child routes on the root. Reorders are a single atomic update; you never have to fight index drift.
• kupe_alertmanager_global — singleton-per-tenant resource for the global section: SMTP defaults, slack_api_url, resolve_timeout, etc.

Example

resource "kupe_alertmanager_receiver" "slack" {
  name = "slack"
  body_json = jsonencode({
    slack_configs = [{
      api_url       = var.slack_webhook_url
      channel       = "#alerts"
      send_resolved = true
    }]
  })
}

resource "kupe_alertmanager_receiver" "pagerduty" {
  name = "pagerduty"
  body_json = jsonencode({
    pagerduty_configs = [{
      service_key   = var.pagerduty_service_key
      send_resolved = true
    }]
  })
}

resource "kupe_alertmanager_routes" "main" {
  routes_json = jsonencode([
    {
      matchers       = ["severity=\"critical\""]
      receiver       = "pagerduty"
      group_wait     = "10s"
      repeat_interval = "1h"
    },
    {
      matchers = ["team=\"infra\""]
      receiver = "slack"
    },
  ])

  depends_on = [
    kupe_alertmanager_receiver.slack,
    kupe_alertmanager_receiver.pagerduty,
  ]
}

The receivers, routes, and global section are all stored against a single tenant Alertmanager configuration shared across every cluster in your tenant. To deliver alerts differently per cluster, use a cluster=... matcher on a route — there is no separate per-cluster configuration to manage.

Direct API access

If Terraform isn’t a fit, the same operations are available on the public Kupe API. Authenticate with a Kupe API key and call the /api/v1/tenants/{tenant}/alertmanager/... endpoints. Every read returns an ETag you can pass back as If-Match on subsequent writes for optimistic locking.

See the Kupe API reference for the full surface.

Next steps

• Alerting: define or tune the rules that produce notifications
• Logs: use logs to add context to alert investigations