Skip to main content

Alarm notifications

Netdata Cloud can send centralized alarm notifications to your team whenever a node enters a warning, critical, or unreachable state. By enabling notifications, you ensure no alarm, on any node in your infrastructure, goes unnoticed by you or your team.

If a node is getting disconnected often or has many alarms, we protect you and your team from alarm fatigue by sending you a flood protection notification. Getting one of these notifications is a good signal of health or performance issues on that node.

Netdata Cloud currently supports email notifications. We're working on additional endpoints and functionality.

Admins must enable alarm notifications for their Space(s). All users in a Space can then personalize their notifications settings from within their account menu.

โ• Centralized alarm notifications from Netdata Cloud is a independent process from Agent notifications. You can enable one or the other, or both, based on your needs. However, the alarms you see in Netdata Cloud are based on those received from the Netdata Agent running on your nodes. If you want to tweak or add new alarms that you see in Netdata Cloud, and receive via centralized alarm notifications, you must configure each node's health watchdog.

Manage alarm notifications for a Space#

To enable notifications for a Space, click Manage Space in the Space management area.

The Manage Space

In the modal, click on the Notifications tab. This menu option is visible only to administrators.

The Space-level management panel for alarm

Click on the toggle to enable or disable a notification method.

Manage alarm notifications per user#

You, and other individual users in your Space, can also enable and disable notification methods.

Click on your profile icon at the top-right of the Cloud UI to open your account menu, then Profile in the dropdown. Click on the Notifications tab in the panel that appears.

The user-level management panel for alarm

Enable or disable notification methods with the toggle buttons.

Select which the notifications you want to receive from each War Room:

  • All alarms and unreachable: Receive notifications for all changes in alarm status: critical, warning, and cleared. In addition, receive notifications for any node that enters an unreachable state.
  • All alarms: Receive notifications for all changes in alarm status: critical, warning, and cleared.
  • Critical only: Receive notifications only for critical alarms.
  • No notifications: Receive no notifications for nodes in this War Room.

If a Space's administrator has disabled notifications, you will see a mesage similar to, "E-mail notifications for this space has been disabled by admin," and your settings have no effect.

Flood protection#

If a node has too many state changes like firing too many alarms or going from reachable to unreachable, Netdata Cloud enables flood protection. As long as a node is in flood protection mode, Netdata Cloud does not send notifications about this node. Even with flood protection active, it is possible to access the node directly, either via Netdata Cloud or the local Agent dashboard at http://NODE:19999.

Anatomy of an alarm notification#

Email alarm notifications show the following information:

  • The Space's name
  • The node's name
  • Alarm status: critical, warning, cleared
  • Previous alarm status
  • Time at which the alarm triggered
  • Chart context that triggered the alarm
  • Name and information about the triggered alarm
  • Alarm value
  • Total number of warning and critical alarms on that node
  • Threshold for triggering the given alarm state
  • Calculation or database lookups that Netdata uses to compute the value
  • Source of the alarm, including which file you can edit to configure this alarm on an individual node

Email notifications also feature a Go to Node button, which takes you directly to the offending chart for that node within Cloud's embedded dashboards.

Here's an example email notification for the ram_available chart, which is in a critical state:

Screenshot of an alarm notification email from Netdata

What's next?#

Netdata Cloud's alarm notifications feature leverages the alarms configuration on each node in your infrastructure. If you'd like to tweak any of these alarms, or even add new ones based on your needs, read our health quickstart.

You can also view active alarms in Netdata Cloud for an instant visualization of the health of your infrastructure.

Reach out

If you need help after reading this doc, feel free to create an issue in the netdata/netdata repo or join our community forum to search for an answer. There's a good chance someone else has already found a solution to the same issue.



Last updated on