Skip to main content

Alert notifications

Netdata Cloud can send centralized alert notifications to your team whenever a node enters a warning, critical, or unreachable state. By enabling notifications, you ensure no alert, on any node in your infrastructure, goes unnoticed by you or your team.

If a node is getting disconnected often or has many alerts, we protect you and your team from alert fatigue by sending you a flood protection notification. Getting one of these notifications is a good signal of health or performance issues on that node.

Netdata Cloud currently supports email notifications. We're working on additional endpoints and functionality.

Admins must enable alert notifications for their Space(s). All users in a Space can then personalize their notifications settings from within their account menu.

Centralized alert notifications from Netdata Cloud is a independent process from notifications from Netdata. You can enable one or the other, or both, based on your needs. However, the alerts you see in Netdata Cloud are based on those streamed from your Netdata-monitoring nodes. If you want to tweak or add new alert that you see in Netdata Cloud, and receive via centralized alert notifications, you must configure each node's alert watchdog.

Manage alert notifications for a Space#

To enable notifications for a Space, click Manage Space in the Space management area.

The Manage Space
button

In the modal, click on the Notifications tab. This menu option is visible only to administrators.

The Space-level management panel for alert
notifications

Click on the toggle to enable or disable a notification method.

Manage alert notifications per user#

You, and other individual users in your Space, can also enable and disable notification methods.

Click on your profile icon at the top-right of the Cloud UI to open your account menu, then Profile in the dropdown. Click on the Notifications tab in the panel that appears.

The user-level management panel for alert
notifications

Enable or disable notification methods with the toggle buttons.

Select which the notifications you want to receive from each War Room:

  • All alerts and unreachable: Receive notifications for all changes in alert status: critical, warning, and cleared. In addition, receive notifications for any node that enters an unreachable state.
  • All alerts: Receive notifications for all changes in alert status: critical, warning, and cleared.
  • Critical only: Receive notifications only for critical alerts.
  • No notifications: Receive no notifications for nodes in this War Room.

If a Space's administrator has disabled notifications, you will see a mesage similar to, "E-mail notifications for this space has been disabled by admin," and your settings have no effect.

Flood protection#

If a node has too many state changes like firing too many alerts or going from reachable to unreachable, Netdata Cloud enables flood protection. As long as a node is in flood protection mode, Netdata Cloud does not send notifications about this node. Even with flood protection active, it is possible to access the node directly, either via Netdata Cloud or the local Agent dashboard at http://NODE:19999.

Anatomy of an alert notification#

Email alarm notifications show the following information:

  • The Space's name
  • The node's name
  • Alarm status: critical, warning, cleared
  • Previous alarm status
  • Time at which the alarm triggered
  • Chart context that triggered the alarm
  • Name and information about the triggered alarm
  • Alarm value
  • Total number of warning and critical alerts on that node
  • Threshold for triggering the given alarm state
  • Calculation or database lookups that Netdata uses to compute the value
  • Source of the alarm, including which file you can edit to configure this alarm on an individual node

Email notifications also feature a Go to Node button, which takes you directly to the offending chart for that node within Cloud's embedded dashboards.

Here's an example email notification for the ram_available chart, which is in a critical state:

Screenshot of an alarm notification email from Netdata
Cloud

What's next?#

Netdata Cloud's alarm notifications feature leverages the alarms configuration on each node in your infrastructure. If you'd like to tweak any of these alarms, or even add new ones based on your needs, read our health quickstart.

You can also view active alarms in Netdata Cloud for an instant visualization of the health of your infrastructure.

Reach out

If you need help after reading this doc, search our community forum for an answer. There's a good chance someone else has already found a solution to the same issue.

Documentation

Community

Last updated on