When you use Netdata to monitor and troubleshoot an entire infrastructure, whether that's dozens or hundreds of systems, you need sophisticated ways of keeping everything organized. You need alarms that adapt to the system's purpose, or whether the parent or child in a streaming setup. You need properly-labeled metrics archiving so you can sort, correlate, and mash-up your data to your heart's content. You need to keep tabs on ephemeral Docker containers in a Kubernetes cluster.
You need host labels: a powerful new way of organizing your Netdata-monitored systems. We introduced host labels in v1.20 of Netdata, and they come pre-configured out of the box.
Let's take a peek into how to create host labels and apply them across a few of Netdata's features to give you more organization power over your infrastructure.
Host labels are defined in
netdata.conf. To create host labels, open that file using
Create a new
[host labels] section defining a new host label and its value for the system in question. Make sure not
to violate any of the host label naming rules.
Once you've written a few host labels, you need to enable them. Instead of restarting the entire Netdata service, you
can reload labels using the helpful
Your host labels will now be enabled. You can double-check these by using
curl http://HOST-IP:19999/api/v1/info to
read the status of your agent. For example, from a VPS system running Debian 10:
You may have noticed a handful of labels that begin with an underscore (
_). These are automatic labels.
When Netdata starts, it captures relevant information about the system and converts them into automatically-generated host labels. You can use these to logically organize your systems via health entities, exporting metrics, parent-child status, and more.
They capture the following:
- Kernel version
- Operating system name and version
- CPU architecture, system cores, CPU frequency, RAM, and disk space
- Whether Netdata is running inside of a container, and if so, the OS and hardware details about the container's host
- Whether Netdata is running inside K8s node
- What virtualization layer the system runs on top of, if any
- Whether the system is a streaming parent or child
If you want to organize your systems without manually creating host tags, try the automatic labels in some of the features below.
You may have noticed the
_is_child automatic labels from above. Host labels are also now
streamed from a child to its parent node, which concentrates an entire infrastructure's OS, hardware, container,
and virtualization information in one place: the parent.
Now, if you'd like to remind yourself of how much RAM a certain child node has, you can access
http://localhost:19999/host/CHILD_HOSTNAME/api/v1/info and reference the automatically-generated host labels from the
child system. It's a vastly simplified way of accessing critical information about your infrastructure.
⚠️ Because automatic labels for child nodes are accessible via API calls, and contain sensitive information like kernel and operating system versions, you should secure streaming connections with SSL. See the streaming documentation for details. You may also want to use access lists or expose the API only to LAN/localhost connections.
You can also use
_is_child, and any other host labels in both health entities and metrics
exporting. Speaking of which...
You can use host labels to logically organize your systems by their type, purpose, or location, and then apply specific alarms to them.
For example, let's use configuration example from earlier:
You could now create a new health entity (checking if disk space will run out soon) that applies only to any host
Or, by using one of the automatic labels, for only webserver systems running a specific OS:
In a streaming configuration where a parent node is triggering alarms for its child nodes, you could create health entities that apply only to child nodes:
Or when ephemeral Docker nodes are involved:
Of course, there are many more possibilities for intuitively organizing your systems with host labels. See the health documentation for more details, and then get creative!
If you have enabled any metrics exporting via our experimental exporters, any new host
labels you created manually are sent to the destination database alongside metrics. You can change this behavior by
exporting.conf, and you can even send automatically-generated labels on with exported metrics.
You can also change this behavior per exporting connection:
By applying labels to exported metrics, you can more easily parse historical metrics with the labels applied. To learn more about exporting, read the documentation.
Host labels are a brand-new feature to Netdata, and yet they've already propagated deeply into some of its core functionality. We're just getting started with labels, and will keep the community apprised of additional functionality as it's made available. You can also track issue #6503, which is where the Netdata team first kicked off this work.
It should be noted that while the Netdata dashboard does not expose either user-configured or automatic host labels, API queries do showcase this information. As always, we recommend you secure Netdata
- Expose Netdata only in a private LAN
- Enable TLS/SSL for web/API requests
- Put Netdata behind a proxy
If you have issues or questions around using host labels, don't hesitate to file an issue on GitHub. We're excited to make host labels even more valuable to our users, which we can only do with your input.