Squid log collector
Squid
is a caching and forwarding HTTP web proxy.
This module parses Squid
access logs.
Metrics
All metrics have "squidlog." prefix.
Metric | Scope | Dimensions | Units |
---|---|---|---|
requests | global | requests | requests/s |
excluded_requests | global | unmatched | requests/s |
type_requests | global | success, bad, redirect, error | requests/s |
http_status_code_class_responses | global | 1xx, 2xx, 3xx, 4xx, 5xx | responses/s |
http_status_code_responses | global | a dimension per HTTP response code | responses/s |
bandwidth | global | sent | kilobits/s |
response_time | global | min, max, avg | milliseconds |
uniq_clients | global | clients | clients |
cache_result_code_requests | global | a dimension per cache result code | requests/s |
cache_result_code_transport_tag_requests | global | a dimension per cache result delivery transport tag | requests/s |
cache_result_code_handling_tag_requests | global | a dimension per cache result handling tag | requests/s |
cache_code_object_tag_requests | global | a dimension per cache result produced object tag | requests/s |
cache_code_load_source_tag_requests | global | a dimension per cache result load source tag | requests/s |
cache_code_error_tag_requests | global | a dimension per cache result error tag | requests/s |
http_method_requests | global | a dimension per HTTP method | requests/s |
mime_type_requests | global | a dimension per MIME type | requests/s |
hier_code_requests | global | a dimension per hierarchy code | requests/s |
server_address_forwarded_requests | global | a dimension per server address | requests/s |
Log Parsers
Squidlog supports 3 log parsers:
- CSV
- LTSV
- RegExp
RegExp is the slowest among them, but it is very likely you will need to use it if your log format is not default.
Known Fields
These are Squid
log format codes.
Squidlog is aware how to parse and interpret following codes:
field | squid format code | description |
---|---|---|
resp_time | %tr | Response time (milliseconds). |
client_address | %>a | Client source IP address. |
client_address | %>A | Client FQDN. |
cache_code | %Ss | Squid request status (TCP_MISS etc). |
http_code | %>Hs | The HTTP response status code from Content Gateway to client. |
resp_size | %<st | Total size of reply sent to client (after adaptation). |
req_method | %rm | Request method (GET/POST etc). |
hier_code | %Sh | Squid hierarchy status (DEFAULT_PARENT etc). |
server_address | %<a | Server IP address of the last server or peer connection. |
server_address | %<A | Server FQDN or peer name. |
mime_type | %mt | MIME content type. |
In addition, to
make Squid
native log format
csv parsable, squidlog understands these groups of codes:
field | squid format code | description |
---|---|---|
result_code | %Ss/%>Hs | Cache code and http code. |
hierarchy | %Sh/%<a | Hierarchy code and server address. |
Custom Log Format
Custom log format is easy. Use known fields to construct your log format.
- If using CSV parser
Note: can be used only if all known squid format codes are separated by csv delimiter. For example, if you
have %Ss:%Sh
, csv parser cant extract %Ss
and %Sh
from it, and you need to use RegExp parser.
Copy your current log format. Replace all known squid format codes with corresponding known fields. Replaces others with "-".
jobs:
- name: squid_log_custom_csv_exampla
path: /var/log/squid/access.log
log_type: csv
csv_config:
format: '- resp_time client_address result_code resp_size req_method - - hierarchy mime_type'
- If using LTSV parser
Provide fields mapping. You need to map your label names to known fields.
- name: squid_log_custom_ltsv_exampla
path: /var/log/squid/access.log
log_type: ltsv
ltsv_config:
mapping:
label1: resp_time
label2: client_address
...
- If using RegExp parser
Use pattern with subexpressions names. These names should be known by squidlog. We recommend to use https://regex101.com/ to test your regular expression.
jobs:
- name: squid_log_custom_regexp_exampla
path: /var/log/squid/access.log
log_type: regexp
regexp_config:
format: '^[0-9.]+\s+(?P<resp_time>[0-9]+) (?P<client_address>[\da-f.:]+) (?P<cache_code>[A-Z_]+)\/(?P<http_code>[0-9]+) (?P<resp_size>[0-9]+) (?P<req_method>[A-Z]+) [^ ]+ [^ ]+ (?P<hier_code>[A-Z_]+)\/[\da-z.:-]+ (?P<mime_type>[A-Za-z-]+)'
Configuration
Edit the go.d/squidlog.conf
configuration file using edit-config
from the
Netdata config directory, which is typically at /etc/netdata
.
cd /etc/netdata # Replace this path with your Netdata config directory
sudo ./edit-config go.d/squidlog.conf
This module needs only path
to log file if you
use native log format. If
you use custom log format you need to set it manually.
jobs:
- name: squid
path: /var/log/squid/access.log
For all available options, please see the module configuration file.
Troubleshooting
To troubleshoot issues with the squid_log
collector, run the go.d.plugin
with the debug option enabled. The output
should give you clues as to why the collector isn't working.
Navigate to the
plugins.d
directory, usually at/usr/libexec/netdata/plugins.d/
. If that's not the case on your system, opennetdata.conf
and look for theplugins
setting under[directories]
.cd /usr/libexec/netdata/plugins.d/
Switch to the
netdata
user.sudo -u netdata -s
Run the
go.d.plugin
to debug the collector:./go.d.plugin -d -m squid_log
Do you have any feedback for this page? If so, you can open a new issue on our netdata/learn repository.