AWS IP Ranges

Plugin: netflow-plugin Module: aws-ip-ranges

Overview

Annotate network flows with AWS service and region labels from AWS public IP ranges. AWS publishes its current public IP allocations as a single JSON document at https://ip-ranges.amazonaws.com/ip-ranges.json. The file lists every prefix AWS uses for a published service, tagged with the AWS region and the AWS service that owns the prefix (AMAZON, EC2, S3, CLOUDFRONT, API_GATEWAY, ...). This integration fetches the file periodically, transforms it via a jq expression, and labels matching flow records with provider, region, and service tags.

Once configured, traffic to / from AWS shows up in dashboards as tenant=amazon with per-region (region=us-east-1, region=eu-central-1, ...) and per-service (role=s3, role=cloudfront, ...) breakdown. This is complementary to GeoIP and ASN enrichment: instead of a flow being labeled simply AS16509 Amazon, it becomes labeled with the specific AWS service and region carrying the traffic -- without parsing AS-name strings.

The file is public; no AWS credentials are needed.

Schema of ip-ranges.json:

• top-level: syncToken, createDate, prefixes, ipv6_prefixes
• each entry in prefixes[]: ip_prefix, region, service, network_border_group
• each entry in ipv6_prefixes[]: ipv6_prefix, region, service, network_border_group

For the cross-cutting concepts (how multiple network sources merge, the lookup priority vs static networks config, TLS verification rules, the jq engine and its expected output schema, fetch loop and retry/backoff), see Network Identity.

The plugin issues a periodic GET to https://ip-ranges.amazonaws.com/ip-ranges.json, parses the JSON body, runs the configured jq transform via the jaq library, and merges the resulting prefix-labeled rows into the network-attributes trie shared by all enrichment sources.

AWS does not publish a fixed refresh cadence; the file is updated whenever the AWS IP space changes (typically several times per day) and the syncToken / createDate fields advance on every change. AWS recommends polling no faster than the file actually changes; daily is enough for most flow-attribution use cases. The plugin floors the configured interval at 60s.

This integration is only supported on the following platforms:

• Linux

This integration supports multiple instances configured side-by-side.

Default Behavior

Auto-Detection

Disabled by default. Add an entry under enrichment.network_sources to enable.

Limits

One full AWS prefix document is fetched per refresh. Resource use scales with the number of AWS prefixes selected by your transform and the refresh interval.

Performance Impact

One HTTPS request per refresh interval plus a jq transform over the AWS prefix document. Runtime enrichment does prefix matching for source and destination IPs, and cost scales with the number of loaded network-source records.

Setup

Prerequisites

Outbound HTTPS to AWS

The agent host must be able to reach https://ip-ranges.amazonaws.com/ip-ranges.json. No AWS credentials are needed -- the file is public.

Configuration

Options

Add a named entry under enrichment.network_sources. The map key (e.g. aws) is the source name; it is used in plugin logs but is not automatically attached to flow records -- the record labels come entirely from your transform output (tenant, region, role, name, site, city, country, state, asn, asn_name).

The expected transform output is a stream of objects, each carrying a prefix (CIDR string) plus any subset of the optional attribute fields. See the cross-cutting Network Identity page for the full output schema.

Config options

Option	Description	Default	Required
url	AWS publishes the master file at this URL. Use it as-is unless you mirror it locally for air-gapped environments.	https://ip-ranges.amazonaws.com/ip-ranges.json	yes
interval	How often to fetch the file. AWS updates the document whenever its IP space changes (often several times per day, sometimes more); daily is enough for most uses. The configured value is floored at 60s.	60s	no
timeout	Per-request timeout for the HTTPS GET.	60s	no
method	HTTP method. AWS serves the file via GET; leave at the default.	GET	no
headers	Extra HTTP headers added to the request. Not required for the public AWS URL; only needed if you front the file behind your own authenticated mirror.	{}	no
transform	jq expression (compiled by jaq) that converts the AWS JSON into a stream of {prefix, ...} objects. The default . does not match the AWS schema -- you must supply a real transform (see examples below) or fetches fail because output rows cannot be mapped to the required {prefix, ...} schema.	.	yes

via File

The configuration file name for this integration is netflow.yaml.

You can edit the configuration file using the edit-config script from the Netdata config directory.

cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata
sudo ./edit-config netflow.yaml

Examples

Tag all AWS prefixes by region and service (IPv4 + IPv6)

Recommended starting point. Sets tenant=amazon, region=<aws-region>, role=<service-name-lowercased>. Covers both prefixes[] and ipv6_prefixes[] in one stream so v4 and v6 traffic are tagged consistently.

enrichment:
  network_sources:
    aws:
      url: "https://ip-ranges.amazonaws.com/ip-ranges.json"
      interval: 24h
      timeout: 60s
      transform: |
        (.prefixes + .ipv6_prefixes)[] | {
          prefix: (.ip_prefix // .ipv6_prefix),
          tenant: "amazon",
          region: .region,
          role: (.service | ascii_downcase)
        }

AWS S3 only

Filter to a single AWS service for narrower tagging. Useful when you only care about identifying S3 traffic (e.g. egress-cost attribution).

Config

enrichment:
  network_sources:
    aws-s3:
      url: "https://ip-ranges.amazonaws.com/ip-ranges.json"
      interval: 24h
      transform: |
        (.prefixes + .ipv6_prefixes)[]
        | select(.service == "S3")
        | {
            prefix: (.ip_prefix // .ipv6_prefix),
            tenant: "amazon",
            role: "s3",
            region: .region
          }

Use network_border_group as the site label

AWS exposes a network_border_group field that distinguishes Local Zones / Wavelength Zones from the parent Region. Map it to the site attribute if you want that distinction visible in flow dashboards.

Config

enrichment:
  network_sources:
    aws:
      url: "https://ip-ranges.amazonaws.com/ip-ranges.json"
      interval: 24h
      transform: |
        (.prefixes + .ipv6_prefixes)[] | {
          prefix: (.ip_prefix // .ipv6_prefix),
          tenant: "amazon",
          region: .region,
          site: .network_border_group,
          role: (.service | ascii_downcase)
        }

Default transform: "." fails because output rows are missing prefix

The default . returns the raw JSON object, not the per-prefix stream the plugin expects. You must supply a transform that yields one object per prefix (with at least a prefix field). Use the first example above as your starting template.

AWS service name precedence inside one prefix

AWS sometimes lists the same ip_prefix under multiple service entries (e.g. once under AMAZON and again under EC2). The plugin merges all records into a single prefix trie, and the last write wins for a given prefix length. If you want a specific service to take precedence, filter the transform with select(.service != "AMAZON") so the broader catch-all entries are dropped.

Static networks: config silently overrides AWS labels

When a prefix is defined in both this source and enrichment.networks, the static config wins on every non-empty field. This is by design (operator intent overrides imported data) but can surprise operators who expect the remote feed to be authoritative. See the cross-cutting Network Identity page for the full lookup-priority rules.

Empty result from the transform is treated as failure

If the jq filter happens to produce nothing (bad selector, wrong field name, upstream JSON shape change), the source backs off as if it errored. Check the journal for network-sources warnings: journalctl --namespace netdata | grep network-sources.

TLS verification cannot be disabled

tls.skip_verify: true is rejected by validation. Use tls.ca_file for custom-CA paths if you front AWS behind an internal TLS-terminating proxy with a private CA.