Skip to main content

How to write a Netdata collector in Go

Let's assume you want to write a collector named example.

Steps are:

❗ If you prefer reading the source code, then check the implementation of the example module, it should give you an idea of how things work.

Module Interface#

Every module should implement the following interface:

type Module interface {
Init() bool
Check() bool
Charts() *Charts
Collect() map[string]int64
Cleanup()
}

Init method#

  • Init does module initialization.
  • If it returns false, the job will be disabled.

We propose to use the following template:

// example.go
func (e *Example) Init() bool {
err := e.validateConfig()
if err != nil {
e.Errorf("config validation: %v", err)
return false
}
someValue, err := e.initSomeValue()
if err != nil {
e.Errorf("someValue init: %v", err)
return false
}
e.someValue = someValue
// ...
return true
}

Move specific initialization methods into the init.go file. See suggested module layout.

Check method#

  • Check returns whether the job is able to collect metrics.
  • Called after Init and only if Init returned true.
  • If it returns false, the job will be disabled.

The simplest way to implement Check is to see if we are getting any metrics from Collect. A lot of modules use such approach.

// example.go
func (e *Example) Check() bool {
return len(e.Collect()) > 0
}

Charts method#

❗ Netdata module produces charts, not raw metrics.

Use agent/module package to create them, it contains charts and dimensions structs.

  • Charts returns the charts (*module.Charts).
  • Called after Check and only if Check returned true.
  • If it returns nil, the job will be disabled
  • ⚠️ Make sure not to share returned value between module instances (jobs).

Usually charts initialized in Init and Chart method just returns the charts instance:

// example.go
func (e *Example) Charts() *Charts {
return e.charts
}

Collect method#

  • Collect collects metrics.
  • Called only if Check returned true.
  • Called every update_every seconds.
  • map[string]int64 keys are charts dimensions ids'.

We propose to use the following template:

// example.go
func (e *Example) Collect() map[string]int64 {
ms, err := e.collect()
if err != nil {
e.Error(err)
}
if len(ms) == 0 {
return nil
}
return ms
}

Move metrics collection logic into the collect.go file. See suggested module layout.

Cleanup method#

  • Cleanup performs the job cleanup/teardown.
  • Called if Init or Check fails, or we want to stop the job after Collect.

If you have nothing to clean up:

// example.go
func (Example) Cleanup() {}

Module Layout#

The general idea is to not put everything in a single file.

We recommend using one file per logical area. This approach makes it easier to maintain the module.

Suggested minimal layout:

FilenameContains
module_name.goModule configuration, implementation and registration.
charts.goCharts, charts templates and constructor functions.
init.goInitialization methods.
collect.goMetrics collection implementation.
module_name_test.goPublic methods/functions tests.
testdata/Files containing sample data.

File module_name.go#

❗ See the example example.go.

Don't overload this file with the implementation details.

Usually it contains only:

File charts.go#

❗ See the example: charts.go.

Put charts, charts templates and charts constructor functions in this file.

File init.go#

❗ See the example: init.go.

All the module initialization details should go in this file.

  • make a function for each value that needs to be initialized.
  • a function should return a value(s), not implicitly set/change any values in the main struct.
// init.go
// Prefer this approach.
func (e Example) initSomeValue() (someValue, error) {
// ...
return someValue, nil
}
// This approach is ok too, but we recommend to not use it.
func (e *Example) initSomeValue() error {
// ...
m.someValue = someValue
return nil
}

File collect.go#

❗ See the example: collect.go.

This file is the entry point for the metrics collection.

Feel free to split it into several files if you think it makes the code more readable.

Use collect_ prefix for the filenames: collect_this.go, collect_that.go, etc.

// collect.go
func (e *Example) collect() (map[string]int64, error) {
collected := make(map[string])int64
// ...
// ...
// ...
return collected, nil
}

File module_name_test.go#

❗ See the example: example_test.go.

if you have no experience in testing we recommend starting with testing package documentation.

we use assert and require packages from github.com/stretchr/testify library, check their documentation.

Testing is mandatory.

  • test only public functions and methods (New, Init, Check, Charts, Cleanup, Collect).
  • do not create a test function per a case, use table driven tests . Prefer map[string]struct{ ... } over []struct{ ... }.
  • use helper functions to prepare test cases to keep them clean and readable.

Directory testdata/#

Put files with sample data in this directory if you need any. Its name should be testdata.

Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata".

Helper packages#

There are some helper packages for writing a module.

Reach out

If you need help after reading this doc, search our community forum for an answer. There's a good chance someone else has already found a solution to the same issue.

Documentation

Community

Edit this page
Last updated on