StackPulse agent alert notifications
StackPulse is a software-as-a-service platform for site reliability engineering. It helps SREs, DevOps Engineers and Software Developers reduce toil and alert fatigue while improving reliability of software services by managing, analyzing and automating incident response activities.
Sending Netdata alert notifications to StackPulse allows you to create smart automated response workflows (StackPulse playbooks) that will help you drive down your MTTD and MTTR by performing any of the following:
- Enriching the incident with data from multiple sources
- Performing triage actions and analyzing their results
- Orchestrating incident management and notification flows
- Performing automatic and semi-automatic remediation actions
- Analyzing incident data and remediation patterns to improve reliability of your services
To send the notification you need:
- Create a Netdata integration in the
StackPulse Administration Portal, and copy the
- On your node, navigate to
/etc/netdata/and run the following command:
$ ./edit-config health_alarm_notify.conf
- Set the
EndpointURL you copied earlier:
- Now restart Netdata using
sudo systemctl restart netdata, or the appropriate method for your system. When your node creates an alert, you can see the associated notification on your StackPulse Administration Portal
React to alerts with playbooks
StackPulse allow users to create
Playbooks giving additional information about events that happen in specific
scenarios. For example, you could create a Playbook that responds to a "low disk space" alert by compressing and
cleaning up storage partitions with dynamic data.
Create Playbooks for Netdata alerts
To create a Playbook, you need to access the StackPulse Administration Portal. After the initial setup, you need to access the TRIGGER tab to define the scenarios used to trigger the event. The following variables are available:
Hostname: The host that generated the event.
Chart: The name of the chart.
OldValue: The previous value of the alert.
Value: The current value of the alert.
Units: The units of the value.
OldStatus: The previous status: REMOVED, UNINITIALIZED, UNDEFINED, CLEAR, WARNING, CRITICAL.
State: The current alert status, the acceptable values are the same of
Alarm: The name of the alert, as given in Netdata's health.d entries.
Date: The timestamp this event occurred.
Duration: The duration in seconds of the previous alert state.
NonClearDuration: The total duration in seconds this is/was non-clear.
Description: A short description of the alert copied from the alert definition.
CalcExpression: The expression that was evaluated to trigger the alert.
CalcParamValues: The values of the parameters in the expression, at the time of the evaluation.
TotalWarnings: Total number of alerts in WARNING state.
TotalCritical: Total number of alerts in CRITICAL state.
ID: The unique id of the alert that generated this event.
For more details how to create a scenario, take a look at the StackPulse documentation.
Do you have any feedback for this page? If so, you can open a new issue on our netdata/learn repository.