Creating Alerts
To create an alert, navigate to the alerts page in Outlyer and click the Create new button in the top left.
Pass the popover a required Title and add an optional description. You may also add labels to your alerts to make the lists easier to filter
You will now be presented with two panels, the Alerts Criteria and the Actions
Alerts Criteria
Click on the top right to create a new criteria. Here you can chose from the following three options:
Host Status Criteria
Use labels to target a specific host or set of hosts. You can use multiple labels to filter in on a set, where labels the host will be present in ALL labels specified.
Set the Critical Threshold level: after how many critical returns should any actions be triggered.
Set the Warning Threshold level: after how many warning returns should any actions be triggered.
Set the Recovery Threshold level: after how many normal returns should a recover action be triggered.
It is recommended to set critical to 5, warning to 4 and recovery to 1. With a default host check time of 30 seconds, this would mean a host would be triggered as warning after 2 minutes, marked as critical after 2mins 30 and recovered as soon as it returns to normal.
You must set a name for this alert criteria on the description tab, before saving. You may also set an optional description.
Service Status Criteria
Choose a service to alert on. By default, the service box will return you a list of all scheduled service checks available.
You can filter this list by using labels. Only services action against chosen labels will appear in the service box.
You should use labels to target a specific set of hosts with a service check scheduled against them. For example, you may have a generic Nginx service check against all webserver hosts, but you may just want to target those hosts that serve your API and use their api label appropriately.
Set the Critical Threshold level: after how many critical returns should any actions be triggered.
Set the Warning Threshold level: after how many warning returns should any actions be triggered.
Set the Recovery Threshold level: after how many normal returns should a recover action be triggered.
It is recommended to set critical to 5, warning to 4 and recovery to 1. With a default host check time of 30 seconds, this would mean a host would be triggered as warning after 2 minutes, marked as critical after 2mins 30 and recovered as soon as it returns to normal.
You must set a name for this alert criteria on the description tab, before saving. You may also set an optional description.
Metric Criteria
Selecting the Time Range will add a real-time preview of any chosen metric.
Chose a query, from the bottom panel to select the metric for this alert.
You should use labels to target a specific set of hosts with a service check scheduled against them. Multiple labels will further constrain the set of hosts the metric comes from.
Set the Summarisation to use on the metric. For more information see the Consolidation Functions section of the Analytics documentation.
Choose thresholds at which to trigger critical and warning actions and the time period the metric is in this state before triggering.
Thresholds can be triggered either above or below a certain value. When you add them, a red (critical) and orange (warning) line will be added to the graph to show the relation to the real-time value of the metric
You must set a name for this alert criteria on the description tab, before saving. You may also set an optional description.
Actions
Click on the right of the actions area to create a new criteria.
For each action you can choose the level at which the action occurs
-
Critical - will only action the alerts when a criteria meets the critical level.
-
Warning - will action when the warning level meets warning levels and again when critical is breached.
Now you can chose from the following three options:
Email Action
Fill in the box a list of email addresses to send an alert to when triggered. For each email address, you will need to click the address, for format validation to occur. You can enter multiple addresses.
Click Create to finish and save the action.
Slack Action
Choose a name for this action.
Add the Slack webhook URL that this alert will trigger.
To get this URL from slack:
- visit the Incoming Webhooks page in the Slack App Directory
- Click Add Configuration
- Choose the channel for the Incoming Webhook
- Click Save Settings to finish
- Copy the resultant URL into Outlyer
Click Create to finish and save the action.
Webhook Action
Choose a name for this action.
Add the URL for the webhook action.
Add a payload to send to the webhook.
More will be coming on this action soon!
Click Create to finish and save the action.
