Practice · Symptom-based alerting

What is symptom-based alerting?

Symptom-based alerting fires on what users actually feel, such as high latency or error rate, rather than on internal causes like a single restart or CPU spike. It produces fewer, more meaningful pages, because every alert maps to real user pain worth waking someone for.

How to get started with Symptom-based alerting

  1. 1 Audit current alerts and tag each one as a symptom (users feel it) or a cause (internal detail).
  2. 2 Replace cause alerts that page with SLO burn-rate alerts on latency and errors.
  3. 3 Demote cause signals like CPU or restarts to dashboards and tickets.
  4. 4 Review every page in a weekly on-call handoff and delete alerts nobody acted on.

The common mistake

What is the most common mistake with Symptom-based alerting?

Paging on every restart, CPU spike, and full disk. The noise trains on-call to ignore the pager, including the page that matters.

See it in action

On-call Resolved calmly

The Page That Never Came

A week on call with zero pages. Not because nothing happened, but because error budgets, symptom-based alerts, and ruthless noise pruning meant only the things worth waking for ever woke anyone.

Related practices

All 13 practices →