Service discovery

Service discovery extracts all the potentially useful information from different sources, converts it to the configurations and exports them to the different destinations.

Pipeline#

The service discovery pipeline has four jobs:

  • discovery: dynamically discovers targets.
  • tag: tags discovered targets.
  • build: creates configuration files from the targets.
  • export: exports configurations.

Routing in a job and between jobs based on tags and selector.

Pipeline configuration:

name: <name>
discovery: <discovery_config>
tag: <tag_config>
build: <build_config>
export: <export_config>

Tags and selectors#

Tag, build and export jobs have selector, the pipeline routes a target/config to the job only if its tags matches job selectors.

Both tags and selector are just lists of words.

A word must match the regex ^[a-zA-Z][a-zA-Z0-9=_.]*$.

Tags special cases:

  • -word: the word will be removed on tags merging.

Selectors special cases:

  • !word: should not contain the word.
  • word|word|word: should contain any word.

Discovery#

Discovery job dynamically discovers targets using one of the supported service-discovery mechanisms.

Supported mechanisms:

Discovery configuration:

k8s:
- <kubernetes_discovery_config>

Kubernetes#

Kubernetes discoverer retrieves targets from Kubernetes' REST API. It always stays synchronized with the cluster state.

Kubernetes discovery configuration options:

# Mandatory. Tags to add to all discovered targets.
tags: <tags>
# Mandatory. The Kubernetes role of entities that should be discovered.
role: <role>
# Optional. Discover only targets that exist on the same node as service-discovery.
# This option works only for 'pod' role and it requires MY_NODE_NAME env variable to be set.
local_mode: <boolean>
# Optional. If omitted, all namespaces are used.
namespaces:
- <namespace>

One of the following role types can be configured to discover targets:

  • pod
  • service

Pod Role#

The pod role discovers all pods and exposes their containers as targets. For each declared port of a container, it generates single target. If there is no declared port it generates one target with empty Port, PortName and PortProtocol fields.

Available pod target fields:

NameTypeValue
TUIDstringNamespace_Name_ContName_PortProtocol_Port
AddressstringPodIP:Port
Namespacestringpod.metadata.namespace
Namestringpod.metadata.name
Annotationsmap[string]stringpod.metadata.annotations
Labelsmap[string]stringpod.metadata.labels
NodeNamestringpod.spec.nodeName
PodIPstringpod.status.podIP
ContNamestringpod.spec.containers.name
Imagestringpod.spec.containers.image
Envmap[string]stringpod.spec.containers.env + pod.spec.containers.envFrom
Portstringpod.spec.containers.ports.containerPort
PortNamestringpod.spec.containers.ports.name
PortProtocolstringpod.spec.containers.ports.protocol

Service Role#

The service role discovers a target for each service port for each service.

Available service target fields:

NameTypeValue
TUIDstringNamespace_Name_PortProtocol_Port
AddressstringName.Namespace.svc:Port
Namespacestringsvc.metadata.namespace
Namestringsvc.metadata.name
Annotationsmap[string]stringsvc.metadata.annotations
Labelsmap[string]stringsvc.metadata.labels
Portstringpod.spec.containers.ports.containerPort
PortNamestringpod.spec.containers.ports.name
PortProtocolstringpod.spec.containers.ports.protocol
ClusterIPstringsvc.spec.clusterIP
ExternalNamestringsvc.spec.externalName
Typestringsvc.spec.ports.type

Tag#

Tag job tags targets discovered by discovery job. Its purpose is service identification.

Configuration is a list of tag rules:

- <tag_rule_config>

Tag rule configuration options:

# Mandatory. Routes targets to this tag rule with tags matching this selector.
selector: <selector>
# Mandatory. Tags to merge with the target tags if at least on of the match rules matches.
tags: <tags>
# Mandatory. Match rules, at least one should be defined.
match:
# Optional. Routes targets to this match rule with tags matching this selector.
- selector: <selector>
# Mandatory. Tags to merge with the target tags if this rule expression evaluates to true.
tags: <tags>
# Mandatory. Match expression.
expr: <expression>

Match expression evaluation result should be true or false.

Expression syntax is go-template.

Available functions#

Custom functions:

  • glob reports whether arg1 matches the shell file name pattern.
  • re reports whether arg1 contains any match of the regular expression pattern.

All these functions accepts two or more arguments, returning in effect:

func(arg1, arg2) || func(arg1, arg3) || func(arg1, arg4) ...

Build#

Build job creates configurations from targets.

Configuration is a list of build rules:

- <build_rule_config>

Build rule configuration options:

# Mandatory. Routes targets to this rule with tags matching this selector.
selector: <selector>
# Mandatory. Tags to add to all built by this rule configurations.
tags: <tags>
# Mandatory. Apply rules, at least one should be defined.
apply:
# Mandatory. Routes targets to this apply rule with tags matching this selector.
- selector: <selector>
# Optional. Tags to add to configurations built by this apply rule.
tags: <tags>
# Mandatory. Configuration template.
template: <template>

Template syntax is go-template.

Available functions#

Custom functions:

  • glob reports whether arg1 matches the shell file name pattern.
  • re reports whether arg1 contains any match of the regular expression pattern.

All these functions accepts two or more arguments, returning in effect:

func(arg1, arg2) || func(arg1, arg3) || func(arg1, arg4) ...

Export#

Export job exports configurations built by build job.

Supported exporters:

  • file

Export configuration:

file:
- <file_exporter_config>

File#

File exporter writes configurations to a specific file.

# Mandatory. Routes configurations to this exporter with tags matching this selector.
selector: <selector>
# Mandatory. Absolute path to a file.
filename: <filename>

Troubleshooting#

Service-discovery has debug mode and special stdout exporter which is enabled only when it's running from the terminal.

CLI:

Usage:
sd [OPTION]...
Application Options:
--config-file= Configuration file path
--config-map= Configuration ConfigMap (name:key)
-d, --debug Debug mode
Help Options:
-h, --help Show this help message
Last updated on