Skip to main content
Version: 1.37.x

Kubelet monitoring with Netdata

Kubelet is an agent that runs on each node in the cluster. It makes sure that containers are running in a pod.

This module will monitor one or more kubelet instances, depending on your configuration.

Metrics

All metrics have "k8s_kubelet." prefix.

MetricScopeDimensionsUnits
apiserver_audit_requests_rejectedglobalrejectedrequests/s
apiserver_storage_data_key_generation_failuresglobalfailuresevents/s
apiserver_storage_data_key_generation_latenciesglobal5µs, 10µs, 20µs, 40µs, 80µs, 160µs, 320µs, 640µs, 1280µs, 2560µs, 5120µs, 10240µs, 20480µs, 40960µs, +Infobserves/s
apiserver_storage_data_key_generation_latencies_percentglobal5µs, 10µs, 20µs, 40µs, 80µs, 160µs, 320µs, 640µs, 1280µs, 2560µs, 5120µs, 10240µs, 20480µs, 40960µs, +Infpercentage
apiserver_storage_envelope_transformation_cache_missesglobalcache missesevents/s
kubelet_containers_runningglobaltotalrunning_containers
kubelet_pods_runningglobaltotalrunning_pods
kubelet_pods_log_filesystem_used_bytesglobala dimension per namespace and podB
kubelet_runtime_operationsglobala dimension per operation typeoperations/s
kubelet_runtime_operations_errorsglobala dimension per operation typeerrors/s
kubelet_docker_operationsglobala dimension per operation typeoperations/s
kubelet_docker_operations_errorsglobala dimension per operation typeerrors/s
kubelet_node_config_errorglobalexperiencing_errorbool
kubelet_pleg_relist_interval_microsecondsglobal0.5, 0.9, 0.99microseconds
kubelet_pleg_relist_latency_microsecondsglobal0.5, 0.9, 0.99microseconds
kubelet_token_requestsglobaltotal, failedtoken_requests/s
rest_client_requests_by_codeglobala dimension per HTTP status coderequests/s
rest_client_requests_by_methodglobala dimension per HTTP methodrequests/s
volume_manager_total_volumesvolume manageractual, desiredstate

Configuration

Edit the go.d/k8s_kubelet.conf configuration file using edit-config from the Netdata config directory, which is typically at /etc/netdata.

cd /etc/netdata # Replace this path with your Netdata config directory
sudo ./edit-config go.d/k8s_kubelet.conf

Needs only url to kubelet metric-address. Here is an example for 2 instances:

jobs:
- name: local
url: http://127.0.0.1:10255/metrics

- name: remote
url: http://203.0.113.10:10255/metrics

For all available options please see module configuration file.

Troubleshooting

To troubleshoot issues with the k8s_kubelet collector, run the go.d.plugin with the debug option enabled. The output should give you clues as to why the collector isn't working.

  • Navigate to the plugins.d directory, usually at /usr/libexec/netdata/plugins.d/. If that's not the case on your system, open netdata.conf and look for the plugins setting under [directories].

    cd /usr/libexec/netdata/plugins.d/
  • Switch to the netdata user.

    sudo -u netdata -s
  • Run the go.d.plugin to debug the collector:

    ./go.d.plugin -d -m k8s_kubelet

Was this page helpful?

Contribute