Generic JSON-over-HTTP IPAM

Plugin: netflow-plugin Module: generic-ipam

Overview

Annotate network flows with network-identity labels from any JSON-over-HTTP IPAM or CMDB endpoint. The plugin does not care WHO produces the JSON -- it cares about the JSON shape after your transform (a jq expression) runs over it. Examples: Infoblox WAPI, BlueCat REST API, phpIPAM, ServiceNow CMDB queries, an internal aggregator, a Lambda function, a static file served from S3, a hand-rolled CMDB.

This card is also the reference description of how all the other Network Identity cards work under the hood. AWS IP Ranges, GCP IP Ranges, Azure IP Ranges and NetBox are all special-cased instances of this same mechanism: each is just a known URL plus a recommended jq transform. Read this card to understand the generic shape, then read the per-source cards for the URL and transform that fit that specific provider.

For the full network-identity concept (merge order vs GeoIP and static, what fields you can populate, why TLS verification cannot be disabled), see Network Identity.

The plugin issues a periodic HTTP request (GET by default, POST optionally) to your configured URL with the headers you specify, parses the response as JSON, runs your transform jq expression (compiled by the jaq library) over the parsed body, and merges the resulting per-prefix rows into the network-attributes trie.

The same flow applies to every JSON-over-HTTP source; the integration-specific part is the URL and the transform expression.

This integration is only supported on the following platforms:

• Linux

This integration supports multiple instances configured side-by-side.

Default Behavior

Auto-Detection

Disabled by default. Add a named entry under enrichment.network_sources for each IPAM/CMDB/endpoint you want to import.

Limits

Resource use scales with response size, transform complexity, refresh interval, and the number of emitted prefixes. Empty transform output is treated as a fetch failure.

Performance Impact

One HTTP request per refresh interval plus a jq transform over the response. Runtime enrichment does prefix matching for source and destination IPs, and cost scales with the number of loaded network-source records.

Setup

Prerequisites

An HTTP/HTTPS endpoint returning JSON

The endpoint must respond with a parseable JSON document (the plugin sets Accept: application/json). Only GET and POST are accepted as request methods. There is no pagination, no cursor following, no Link: rel=next handling -- the fetch is one-shot per cycle. If your IPAM paginates, expose a bulk endpoint or wrap it in a server-side aggregator that returns the full list at one URL.

Authentication via headers

The plugin has no built-in OAuth flow, basic-auth helper, or token refresh. Whatever the API needs (bearer tokens, API keys, custom header names, basic-auth realms) goes into the headers: map. The header map is an arbitrary string-to-string mapping, so any single-shot scheme works. For short-lived tokens, refresh them outside Netdata and reload the plugin config.

A POST endpoint must accept an empty body

When method: POST is configured, the plugin sends the request with the configured headers but no request body. If your CMDB requires a JSON query body to return prefixes, wrap it server-side with an endpoint that accepts GET (or POST with no body) and returns the full prefix set.

Configuration

Options

Add a named entry under enrichment.network_sources. Unknown keys cause a config error.

Config options

Option	Description	Default	Required
url	HTTP/HTTPS endpoint. Required (a non-empty URL is enforced by validation).		yes
method	HTTP method. Only GET and POST are accepted; anything else fails validation. Note that POST is sent with no request body.	GET	no
headers	Map of additional HTTP request headers. Use this for any authentication scheme (Authorization: Bearer ..., token: ..., custom API-key headers, basic-auth realms encoded explicitly, etc.). Values are passed through verbatim.	{}	no
interval	Refresh cadence. Values below 60 seconds are accepted, but refreshes still run no faster than once per minute. Pick the value that matches how often your IPAM actually changes (5-15 minutes for a curated CMDB; daily for slow-moving prefix lists).	60s	no
timeout	Per-request timeout. Must be greater than 0.	60s	no
proxy	Whether to honor the system HTTP/HTTPS proxy environment variables. Set to false to bypass the proxy for this source (useful when the IPAM is on the internal network and the proxy is for outbound traffic only).	true	no
tls.enable	Enables custom TLS settings (custom CA bundle, mTLS client certificate). Must be true whenever any of tls.ca_file, tls.cert_file, tls.key_file is set.	false	no
tls.verify	Certificate verification toggle. Setting this to false is rejected by validation -- network-identity data flows directly into security-relevant enrichment, so MITM-able responses are not allowed. Use tls.ca_file to trust internal CAs instead.	true	no
tls.skip_verify	Legacy alias for the same intent as tls.verify: false. Setting this to true is rejected by validation. Use tls.ca_file.	false	no
tls.ca_file	PEM file with the CA bundle to trust for this endpoint (instead of, or in addition to, the system roots). The recommended way to talk to an internal IPAM behind your own PKI.		no
tls.cert_file	PEM file with the client certificate, for mTLS-protected endpoints.		no
tls.key_file	PEM file with the client private key. Required only when the key is in a separate file from the certificate; if omitted, the cert file is reused. tls.cert_file must also be set whenever tls.key_file is set.		no
transform	jq expression compiled by jaq. Receives the entire parsed JSON body once and must produce a stream of objects, where each object has at minimum a prefix field (a CIDR string -- IPv4/len or IPv6/len) plus any of the optional attribute fields: name, role, site, region, country, state, city, tenant, asn, asn_name. Any field not produced is treated as empty. The transform compiles at startup -- a syntax error fails the config load. An empty stream at runtime is treated as a fetch failure and triggers backoff.	.	yes

via File

The configuration file name for this integration is netflow.yaml.

You can edit the configuration file using the edit-config script from the Netdata config directory.

cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata
sudo ./edit-config netflow.yaml

Examples

Flat list with {prefix, name, env}

The simplest IPAM shape: a top-level array of subnets, each with a CIDR string and a few labels. Maps env -> tenant and the name -> NET_NAME.

enrichment:
  network_sources:
    custom_ipam:
      url: "https://ipam.corp.example/api/v1/prefixes"
      headers:
        Authorization: "Bearer abcdef0123456789"
      interval: 5m
      transform: |
        .[] | {
          prefix: .prefix,
          name: .name,
          tenant: .env
        }

phpIPAM with API token

phpIPAM exposes /api/<APP>/subnets/. Replace <APP> with your phpIPAM application name. phpIPAM splits the address into a base and a mask -- the transform reassembles them into a CIDR string.

Config

enrichment:
  network_sources:
    phpipam:
      url: "https://ipam.example/api/netdata/subnets/"
      headers:
        token: "abcdef..."
      interval: 10m
      transform: |
        .data[] | {
          prefix: (.subnet + "/" + (.mask|tostring)),
          name: .description,
          tenant: (.custom_tenant // ""),
          site: (.location.name // "")
        }

Hierarchical CMDB export

A CMDB that returns nested objects -- top-level sites, each with a list of vlans, each with a list of prefixes. Flattens to one row per prefix, inheriting site and role labels from the parent.

Config

enrichment:
  network_sources:
    cmdb:
      url: "https://cmdb.example/export/networks.json"
      headers:
        Authorization: "Bearer <CMDB_TOKEN>"
      interval: 15m
      transform: |
        .sites[] as $site
        | $site.vlans[] as $vlan
        | $vlan.prefixes[]
        | {
            prefix: .cidr,
            name: .label,
            site: $site.name,
            region: $site.region,
            role: $vlan.role,
            tenant: $vlan.tenant
          }

Internal IPAM behind mTLS

When the IPAM is fronted by your internal PKI and requires a client certificate. tls.enable: true activates the custom TLS path; tls.verify must remain true (cannot be disabled).

Config

enrichment:
  network_sources:
    corp_ipam:
      url: "https://ipam.corp/api/networks"
      tls:
        enable: true
        ca_file: /etc/netdata/ssl/corp-ca.pem
        cert_file: /etc/netdata/ssl/netdata.crt
        key_file: /etc/netdata/ssl/netdata.key
      interval: 10m
      transform: |
        .[] | {
          prefix: .cidr,
          name: .label,
          tenant: .tenant
        }

Source with the system proxy bypassed

When the agent host has an outbound HTTP proxy configured but the IPAM lives on the internal network and should be reached directly.

Config

enrichment:
  network_sources:
    internal_ipam:
      url: "https://ipam.internal/api/networks"
      proxy: false
      headers:
        Authorization: "Bearer ..."
      interval: 5m
      transform: |
        .[] | { prefix: .cidr, name: .label }

Endpoint requires pagination

The plugin does not paginate. Either raise the page size in the URL to cover your full inventory, or wrap the endpoint with a server-side aggregator that returns all results at one URL. There is no built-in Link: rel=next follower.

POST endpoint requires a request body

The plugin's POST request is sent with no body. If your CMDB requires a JSON query body to return prefixes, wrap it server-side with an endpoint that accepts GET (or accepts POST with no body) and returns the full prefix set.

TLS verification cannot be disabled

tls.verify: false and tls.skip_verify: true are both rejected during configuration validation. Use tls.ca_file to trust internal CAs. This is deliberate -- network-identity rows feed enrichment used in security investigations and capacity decisions, where silently accepting MITM-able responses would corrupt every downstream analysis.

Empty result back-off

An empty stream from the jq transform is treated as a fetch failure. The source then backs off exponentially -- starting at interval / 10 (floor 1s), doubling on each consecutive failure, and capped at the regular interval. On the next successful non-empty fetch the cadence resets to interval. If your IPAM legitimately has no prefixes (a quiet state), have the upstream return at least one synthetic prefix so the source does not back off.

Refresh appears slower than configured

The fetch loop floors the configured interval at 60 seconds. Configuring interval: 5s does not produce a 5-second loop -- it produces a 60-second loop. Pick a value at or above 60s that matches how often your IPAM actually changes (5-15 minutes is typical for a curated CMDB).

Unknown config keys cause errors

The config struct uses deny_unknown_fields. Typos like headres: or a non-existent option fail config load with a parse error rather than being silently ignored. Check the enrichment.network_sources.<name> schema listed under "Config options".

JSON parse errors are silent in the dashboard

Decode failures (HTTP error, JSON parse error, jq runtime error, schema mismatch on prefix) are logged but do not surface in the dashboard. Watch the Netdata journal for warnings: journalctl --namespace netdata | grep network_sources.

Prefer explicit authorization headers over URL credentials

URLs with embedded credentials (https://user:pass@host) are converted to HTTP Basic authentication by the HTTP client. Prefer headers: for clarity and to avoid storing credentials in URLs -- e.g. headers: { Authorization: "Basic dXNlcjpwYXNz" } for HTTP basic-auth.