BIND can report a healthy exporter scrape while the recursive lookups it serves are failing. On our IPv4-only lab VM, bind_exporter showed bind_up 1 while six of seven cold-cache lookups came back SERVFAIL, because named kept trying root servers over IPv6. Only a client-side DNS probe caught it, so the setup below pairs named’s own counters with a probe that asks questions the way your clients do.
This guide sets up DNS monitoring with Prometheus and Grafana on a BIND 9 server. You get the statistics channel, bind_exporter, blackbox DNS probes, an importable dashboard and alerts we fired by stopping named and its exporter. Everything was run on Debian 13 in October 2026 with BIND 9.20.29, Prometheus 3.13.4 LTS, bind_exporter 0.8.0, blackbox_exporter 0.26.0 and Grafana 13.2.4. We did not retest Unbound or PowerDNS, although the probes work against any DNS server.
How DNS monitoring with Prometheus fits together
Everything runs on one Debian 13 host in this guide, named mon01 at 10.0.1.50. BIND keeps its counters behind a statistics channel bound to 127.0.0.1, so bind_exporter has to run on the DNS server itself. The blackbox exporter can live anywhere that reaches port 53, which makes it the piece you’d move to a second machine once you monitor more than one server. Prometheus scrapes all of them every 15 seconds and Grafana reads from Prometheus:
The two exporters answer different questions. bind_exporter tells you how many queries arrived, which rcodes went out and how the resolver cache behaves. The blackbox probe asks the server for a SOA record and for an outside name, exactly like a client would, and reports whether that worked and how long it took. You want both, because each one misses failures the other catches.
Prerequisites
Prometheus sizing depends on active series multiplied by retention, not on how busy the DNS server is. In our final replay of this guide, Prometheus held 2,374 active series and its cgroup used about 40 MiB, named used 37 MiB and each exporter about 11 MiB. Grafana is the heavy part: 355 MiB right after setup on the replay VM, and about 870 MiB after a session of dashboard work on our first lab VM, so you should budget RAM for Grafana before anything else. You need the following.
- Debian 13 with sudo access
- BIND 9 built with libxml2 or JSON statistics support (the Debian package has both)
- 2 vCPU and 2 GB RAM for BIND, Prometheus, Grafana and the exporters on one host
- TCP 9090 (Prometheus) and 3000 (Grafana) reachable from your admin network only
- Outbound internet on the DNS server if it also resolves recursively
Our VM had 2 vCPU and 4 GB of RAM, which is a floor for following along, not a production recommendation. A Prometheus server watching dozens of resolvers wants its own machine. If you’re building that skill set from scratch, our list of the best Grafana and Prometheus books covers PromQL and alerting in more depth than any single tutorial can. Already running BIND? The BIND DNS server setup on Debian covers zones, forwarders and ACLs, and this guide only adds monitoring to it.
Set the shell variables
The zone name and the server address appear in the BIND config, the blackbox modules, the Prometheus targets and the test commands, so set them once. Use your own zone and the address clients use to reach the server:
export DNS_ZONE="example.com"
export HOST_IP="10.0.1.50"These variables live only in the current shell, so they are gone the moment you reconnect or switch to sudo -i. If a later command prints an empty zone name or a blank IP, that is the reason, and you fix it by running the two exports again before you continue.
Install Prometheus from the upstream release
Debian 13 packages Prometheus 2.53, and upstream ended support for the 2.53 long-term release on 31 July 2025, according to the Prometheus release cycle page. The 3.x line also brings the new web UI you’ll see in the screenshots. So we install the official release tarball. The exporters still come from Debian, for reasons covered in their own section. For a monitoring server you set up once and leave running, the long-term support line is the better pick: 3.13 is supported until 31 July 2027, while a regular minor release only gets fixes for about six weeks. Pin the LTS line and detect its newest patch release:
PROM_LTS="3.13" # pin: LTS line, supported until 2027-07-31 https://prometheus.io/docs/introduction/release-cycle/
PROM_VER=$(curl -s "https://api.github.com/repos/prometheus/prometheus/releases?per_page=100" | grep -oP "\"tag_name\": \"v\K${PROM_LTS}\.[0-9]+(?=\")" | head -1)
echo "${PROM_VER}"You should see a clean version string on the 3.13 line. An empty line means the GitHub API rate limit kicked in, which happens on shared NAT addresses, so wait a few minutes and run it again. When the next LTS ships, change PROM_LTS and nothing else.
3.13.4Download and unpack the tarball, then create a locked system user and copy the two binaries into place. The URL is for x86_64; on an ARM server use linux-arm64 instead. Prometheus 3 no longer ships the old consoles directories, so older guides that copy them will fail at this step. Only the data directory needs to belong to the prometheus user.
cd /tmp
curl -sLO "https://github.com/prometheus/prometheus/releases/download/v${PROM_VER}/prometheus-${PROM_VER}.linux-amd64.tar.gz"
tar xzf "prometheus-${PROM_VER}.linux-amd64.tar.gz"
sudo useradd --system --no-create-home --shell /usr/sbin/nologin prometheus
sudo install -m 0755 "prometheus-${PROM_VER}.linux-amd64/prometheus" "prometheus-${PROM_VER}.linux-amd64/promtool" /usr/local/bin/
sudo mkdir -p /etc/prometheus /var/lib/prometheus
sudo chown prometheus:prometheus /var/lib/prometheusPrometheus refuses to start without a config file, and the DNS jobs cannot be added until the exporters exist. So you start with a minimal file that only scrapes Prometheus itself and replace it later in the guide:
sudo vim /etc/prometheus/prometheus.ymlPaste this starting config. It sets a 15 second scrape and rule interval, which is the default most dashboards on grafana.com assume, so their rate windows line up with your data.
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: prometheus
static_configs:
- targets: ['localhost:9090']The tarball has no systemd unit, so you write one. Retention is the knob that decides disk use later, because Prometheus keeps every sample for that long whether anyone looks at it or not.
sudo vim /etc/systemd/system/prometheus.serviceAdd the service definition. The ExecReload line matters later, since it lets you apply config changes with systemctl reload instead of a restart that would pause scraping.
[Unit]
Description=Prometheus monitoring server
Wants=network-online.target
After=network-online.target
[Service]
User=prometheus
Group=prometheus
ExecStart=/usr/local/bin/prometheus \
--config.file=/etc/prometheus/prometheus.yml \
--storage.tsdb.path=/var/lib/prometheus \
--storage.tsdb.retention.time=30d
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
[Install]
WantedBy=multi-user.targetCheck the config with promtool before you start the service. Prometheus exits on a bad config, so systemd would otherwise keep restarting it while you wonder why the port never opens. The last command asks the readiness endpoint whether the TSDB is open.
promtool check config /etc/prometheus/prometheus.yml
sudo systemctl daemon-reload
sudo systemctl enable --now prometheus
curl -s --retry 10 --retry-connrefused --retry-delay 1 http://localhost:9090/-/readyThe retry flags matter: in our replay of this guide on a fresh VM, a plain curl right after the start got connection refused (exit code 7), because Prometheus needs a moment to open its TSDB. With retries, the readiness endpoint answers as soon as it can. If it never does, journalctl -u prometheus almost always points at a YAML indentation error:
Checking /etc/prometheus/prometheus.yml
SUCCESS: /etc/prometheus/prometheus.yml is valid prometheus config file syntax
Created symlink '/etc/systemd/system/multi-user.target.wants/prometheus.service' → '/etc/systemd/system/prometheus.service'.
Prometheus Server is Ready.If you’d rather keep Prometheus on its own host and follow a dedicated walkthrough, the Prometheus install guide for Ubuntu and Debian covers that path. Everything below works the same against a remote Prometheus as long as it can reach the exporter ports.
Enable the BIND statistics channel
bind_exporter reads named’s counters over HTTP from the statistics channel, which is off by default, so a stock BIND gives the exporter nothing to read. You enable the channel, add a test zone and confirm the channel answers before any exporter is involved, which keeps the troubleshooting simple. Install BIND and the dig tools first.
sudo apt install -y bind9 bind9-dnsutilsThe exporter’s README asks for a BIND built with libxml2. named lives in /usr/sbin, which is not on a regular user’s PATH on Debian, so call it by full path:
/usr/sbin/named -V | grep -i xmlBoth lines should print a version. A build without libxml2 can still serve the JSON statistics the exporter reads by default, so the real test is the channel check at the end of this section, not this one.
compiled with libxml2 version: 2.9.14
linked to libxml2 version: 20914The statistics-channels statement is a top-level statement of its own, so it goes after the options { ... }; block, never inside it, and you leave everything already in the file alone. Look at what’s there first, without the comments, and check that nobody has added a statistics channel already:
grep -v '^\s*//' /etc/bind/named.conf.options | grep -v '^\s*$'
sudo grep -rn "statistics-channels" /etc/bind/A fresh Debian 13 install holds nothing but the cache directory, and the second command prints nothing. On a server you already run, the options block will carry your forwarders, ACLs and DNSSEC settings, which stay exactly as they are:
options {
directory "/var/cache/bind";
};If the grep found an existing channel, reuse it instead of adding a second one. When it listens on another address or port, point the exporter at it later by adding --bind.stats-url=http://127.0.0.1:<port>/ to the bind_exporter ARGS line, using that channel’s address and port. Otherwise open the file:
sudo vim /etc/bind/named.conf.optionsAdd this block at the very end of the file, below the closing }; of options. The channel listens on 127.0.0.1 only, which matters because it exposes server internals without any authentication:
statistics-channels {
inet 127.0.0.1 port 8053 allow { 127.0.0.1; };
};A test zone gives the exporter and the probes something authoritative to report on. If your server already serves zones, skip the zone file and the registration command below, and make sure DNS_ZONE names one of your zones. The zone file uses relative names, so it works for any zone name:
sudo vim "/etc/bind/db.${DNS_ZONE}"Add the records. The serial uses the date format, so bump it every time you edit the file or secondaries will ignore the change.
$TTL 3600
@ IN SOA ns1 hostmaster (
2026101801 ; serial
3600 ; refresh
900 ; retry
1209600 ; expire
300 ) ; negative cache TTL
@ IN NS ns1
ns1 IN A 10.0.1.50
@ IN A 10.0.1.50
www IN A 10.0.1.50
mail IN A 10.0.1.51
@ IN MX 10 mailRegister the test zone and validate its file. named-checkzone prints the serial it loaded, which tells you the file parsed:
echo "zone \"${DNS_ZONE}\" { type primary; file \"/etc/bind/db.${DNS_ZONE}\"; };" | sudo tee -a /etc/bind/named.conf.local
sudo named-checkzone "${DNS_ZONE}" "/etc/bind/db.${DNS_ZONE}"The tee echoes the line it appended, then the zone check reports the serial:
zone "example.com" { type primary; file "/etc/bind/db.example.com"; };
zone example.com/IN: loaded serial 2026101801
OKEveryone runs the next step, test zone or not. Check the whole config before the restart, because a syntax error otherwise takes the whole server down, then query the zone:
sudo named-checkconf
sudo systemctl restart named
dig +short @127.0.0.1 "www.${DNS_ZONE}" A
dig +short @"${HOST_IP}" "${DNS_ZONE}" MXBoth lookups should return the records from the file (with your own zone, ask for names that exist in it). The second query goes to the server’s LAN address on purpose, because that is the address the blackbox probes use, and a server that only answers on 127.0.0.1 would fail them:
10.0.1.50
10 mail.example.com.Finally, confirm the statistics channel answers. The current bind_exporter reads the JSON flavour by default (--bind.stats-version=json), so test that path rather than the XML root page:
curl -s -o /dev/null -w "%{http_code} %{content_type}\n" http://127.0.0.1:8053/json/v1/serverOur server returned the status code and content type the exporter expects:
200 application/jsonA 200 application/json means the exporter will have data. Connection refused means named did not pick up the new options file, so rerun sudo named-checkconf and check journalctl -u named before you move on.
Install bind_exporter and blackbox_exporter
Debian 13 packages the same bind_exporter version as the latest upstream bind_exporter release, and the project still gets commits even though that release dates from November 2024. The other two exporters lag: Debian has blackbox_exporter 0.26.0 against 0.29.0 upstream and node_exporter 1.9.0 against 1.12.1. Nothing in this guide needs the newer releases, and the packages ship systemd units, a /etc/default file for flags and the prometheus service user, so apt is still the shorter path:
sudo apt install -y prometheus-bind-exporter prometheus-blackbox-exporterBoth services start right away and listen on every interface, which you don’t want. The blackbox exporter in particular will probe any target anyone asks it to, so an open 9115 turns your DNS server into a free probing proxy. With Prometheus on the same host, bind both exporters to 127.0.0.1 by prepending the flag to the ARGS line in their /etc/default files:
sudo sed -i 's|^ARGS="|ARGS="--web.listen-address=127.0.0.1:9119 |' /etc/default/prometheus-bind-exporter
sudo sed -i 's|^ARGS="|ARGS="--web.listen-address=127.0.0.1:9115 |' /etc/default/prometheus-blackbox-exporter
grep ^ARGS /etc/default/prometheus-bind-exporter /etc/default/prometheus-blackbox-exporter
sudo systemctl restart prometheus-bind-exporter prometheus-blackbox-exporter
sleep 2
sudo ss -tlnp | grep -E ":(9115|9119) "The sleep gives the restarted services a moment to bind; without it, our replay caught one of them before it was listening. Both ports should now show 127.0.0.1 instead of *, and a request to http://10.0.1.50:9119/metrics gets connection refused. If Prometheus runs on another machine, use the server’s private address in the flag instead and allow only the Prometheus host to reach it in your firewall:
LISTEN 0 4096 127.0.0.1:9119 0.0.0.0:* users:(("prometheus-bind",pid=3264,fd=3))
LISTEN 0 4096 127.0.0.1:9115 0.0.0.0:* users:(("prometheus-blac",pid=3266,fd=3))The bind exporter’s default stats URL is http://localhost:8053/, which matches the channel we just opened, so it needs no other flags. Ask it for the health metric and the two timestamps BIND reports:
curl -s --retry 5 --retry-connrefused --retry-delay 1 http://localhost:9119/metrics | grep -E "^bind_up|^bind_boot_time|^bind_config_time"bind_up 1 means the exporter read the channel. A 0 here with named running means the URL or the allow list is wrong:
bind_boot_time_seconds 1.792325755e+09
bind_config_time_seconds 1.792325755e+09
bind_up 1The blackbox exporter needs DNS modules. Debian’s default /etc/prometheus/blackbox.yml has HTTP, TCP, gRPC, ICMP and banner-check modules but nothing for DNS. Open it:
sudo vim /etc/prometheus/blackbox.ymlPaste these two modules at the end of the file, indented under the existing modules: key. dns_soa asks for your zone’s SOA with recursion off, which tests the authoritative side. dns_resolve asks for an outside name to test recursion. On an authoritative-only server (recursion no; or a tight allow-recursion), that query gets REFUSED and DnsProbeFailed fires on a healthy server, so leave out the dns_resolve module and its scrape job there. The field names come from the blackbox_exporter configuration reference:
dns_soa:
prober: dns
timeout: 5s
dns:
query_name: "DNS_ZONE_HERE"
query_type: "SOA"
preferred_ip_protocol: "ip4"
recursion_desired: false
valid_rcodes:
- NOERROR
dns_resolve:
prober: dns
timeout: 5s
dns:
query_name: "debian.org"
query_type: "A"
preferred_ip_protocol: "ip4"
valid_rcodes:
- NOERRORThe DNS_ZONE_HERE placeholder is there because a YAML file cannot read shell variables. Swap in your zone, check the file and reload the exporter:
sudo sed -i "s/DNS_ZONE_HERE/${DNS_ZONE}/" /etc/prometheus/blackbox.yml
prometheus-blackbox-exporter --config.check --config.file /etc/prometheus/blackbox.yml
sudo systemctl reload prometheus-blackbox-exporterProbe the zone by hand before you wire it into Prometheus, so you know a failure later is a Prometheus problem and not a DNS one. Every probe returns a set of metrics, and probe_success is the one the alerts key on.
curl -s "http://localhost:9115/probe?module=dns_soa&target=${HOST_IP}:53" | grep -E "^probe_(success|duration_seconds|dns_answer_rrs|dns_serial)"The probe got one SOA record back in well under a millisecond, and it even exposes the zone serial, which is handy for spotting a secondary that stopped transferring:
probe_dns_answer_rrs 1
probe_dns_serial 2.026101801e+09
probe_duration_seconds 0.000548983
probe_success 1That covers the authoritative side. The recursive module is the one that needs outside access, and it is the one that failed in our lab.
Recursive probe returns SERVFAIL on an IPv4-only server
On a server with no IPv6 route, named can answer SERVFAIL to cold recursive lookups until you start it with -4. The summary metrics only showed probe_success 0, so we added debug=true, which makes the exporter print the DNS answer it received and the reason it rejected it.
curl -s "http://localhost:9115/probe?module=dns_resolve&target=${HOST_IP}:53&debug=true" | grep -E "Probe (succeeded|failed)|string_rcode"It failed repeatedly. Across our lab runs without -4, six of the seven cold lookups we captured with debug output returned SERVFAIL, and the one that got through took just over a second. This is the run from our final replay of this guide on a fresh VM:
time=2026-10-18T12:16:04.333Z level=ERROR source=dns.go:122 msg="Rcode is not one of the valid rcodes" module=dns_resolve target=10.0.1.50:53 rcode=2 string_rcode=SERVFAIL valid_rcodes=[0]
time=2026-10-18T12:16:04.333Z level=ERROR source=handler.go:135 msg="Probe failed" module=dns_resolve target=10.0.1.50:53 duration_seconds=0.807079773Meanwhile bind_up stayed at 1, because the statistics channel was fine. named’s own log showed the cause: our VM has no IPv6 default route, but named still tried the IPv6 addresses of the root and TLD servers. Count those messages for the last minute:
sudo journalctl -u named --since "-1min" --no-pager | grep -c "network unreachable"
sudo journalctl -u named --since "-1min" --no-pager | grep "network unreachable" | head -2In that replay, named had logged 92 of them within a minute of the probe. The first two show the pattern:
92
Oct 18 12:15:54 mon01 named[1387]: network unreachable resolving './DNSKEY/IN': 2001:500:12::d0d#53
Oct 18 12:15:54 mon01 named[1387]: network unreachable resolving './DNSKEY/IN': 2001:503:c27::2:30#53Check whether your server has an IPv6 default route. If this prints nothing, it does not, and named should run in IPv4-only mode:
ip -6 route show defaultOn an IPv4-only server, add -4 to named’s startup options in /etc/default/named and restart it. Leave this step out on a host with working IPv6, because it disables IPv6 transport for both queries and answers. The sed only matches Debian’s default line, so the grep after it is your proof that the flag landed:
sudo sed -i 's/^OPTIONS="-u bind"/OPTIONS="-u bind -4"/' /etc/default/named
grep ^OPTIONS /etc/default/named
sudo systemctl restart namedThe grep must show -4. If your file had other options, it prints the old line unchanged, and you add -4 inside the quotes by hand:
OPTIONS="-u bind -4"Flush the cache so the next lookup really is cold, then probe again:
sudo rndc flush
curl -s "http://localhost:9115/probe?module=dns_resolve&target=${HOST_IP}:53&debug=true" | grep -E "Probe (succeeded|failed)|string_rcode"Now the answer is NOERROR and the probe succeeds:
time=2026-10-18T12:16:07.733Z level=INFO source=dns.go:118 msg="Rcode is valid" module=dns_resolve target=10.0.1.50:53 rcode=0 string_rcode=NOERROR
time=2026-10-18T12:16:07.733Z level=INFO source=handler.go:133 msg="Probe succeeded" module=dns_resolve target=10.0.1.50:53 duration_seconds=1.2787261079999999With -4, every flushed-cache probe we ran passed (6 of 6). Cold lookups of debian.org took 1.26 to 1.94 seconds, cached ones answered in 0.34 to 0.42 ms, and named logged zero “network unreachable” lines after the restart. That’s the reason this guide uses both exporters: the server’s own counters can look healthy while clients get SERVFAIL.
Add the DNS scrape jobs to Prometheus
Dashboard 12309, which you import later, also draws CPU, load and memory panels from node_exporter. Without it, three panels stay empty, so install it now while you are adding exporters anyway. Skip the recommended packages: on our VM a plain install pulled in 32 packages, including smartmontools and NVMe and IPMI helpers, against one with --no-install-recommends. Bind it to 127.0.0.1 the same way as the other two:
sudo apt install -y --no-install-recommends prometheus-node-exporter
sudo sed -i 's|^ARGS="|ARGS="--web.listen-address=127.0.0.1:9100 |' /etc/default/prometheus-node-exporter
sudo systemctl restart prometheus-node-exporter
sleep 2
sudo ss -tlnp | grep ":9100 "The listener should show 127.0.0.1 only:
LISTEN 0 4096 127.0.0.1:9100 0.0.0.0:* users:(("prometheus-node",pid=3286,fd=3))Now replace the starter Prometheus config with the full one. The two probe jobs use the standard blackbox relabeling, which turns each target into a target parameter on the probe URL and keeps the DNS server’s address as the instance label:
sudo vim /etc/prometheus/prometheus.ymlUse this config. The node and bind jobs scrape the exporters directly, while the two probe jobs talk to the blackbox exporter on 9115 and pass it your DNS server as the target.
global:
scrape_interval: 15s
evaluation_interval: 15s
rule_files:
- /etc/prometheus/rules/*.yml
scrape_configs:
- job_name: prometheus
static_configs:
- targets: ['localhost:9090']
- job_name: node
static_configs:
- targets: ['localhost:9100']
- job_name: bind
static_configs:
- targets: ['localhost:9119']
- job_name: dns_soa
metrics_path: /probe
params:
module: [dns_soa]
static_configs:
- targets: ['HOST_IP_HERE:53']
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: localhost:9115
- job_name: dns_resolve
metrics_path: /probe
params:
module: [dns_resolve]
static_configs:
- targets: ['HOST_IP_HERE:53']
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: localhost:9115The alert rules referenced by rule_files come in a later section. Create the directory now so Prometheus finds it, fill in the server address and reload:
sudo sed -i "s/HOST_IP_HERE/${HOST_IP}/" /etc/prometheus/prometheus.yml
sudo mkdir -p /etc/prometheus/rules
promtool check config /etc/prometheus/prometheus.yml
sudo systemctl reload prometheusOpen http://10.0.1.50:9090/targets after about 30 seconds. All five jobs should be UP, and the probe jobs should show the DNS server’s address as their instance rather than the exporter’s:
A probe job that’s DOWN means Prometheus cannot reach the blackbox exporter on 9115. A probe job that’s UP with probe_success 0 means the exporter answered but the DNS query failed, which is the case the alerts below catch.
Query BIND metrics with PromQL
bind_exporter publishes counters, so almost every useful query wraps them in rate(). To have numbers worth reading, we pushed the authoritative zone with dnsperf from the same VM. Install it and build a small query file from your zone name, so the test hits records your server is authoritative for instead of measuring cached recursion:
sudo apt install -y dnsperf
printf '%s\n' "www.${DNS_ZONE} A" "${DNS_ZONE} MX" "${DNS_ZONE} SOA" "mail.${DNS_ZONE} A" "ns1.${DNS_ZONE} A" "nohost.${DNS_ZONE} A" > /tmp/queries-auth.txt
cat /tmp/queries-auth.txtOne query per line, name then type. With your own zone, swap the names for records that exist in it. The last line is a name that does not exist, so a sixth of the answers will be NXDOMAIN:
www.example.com A
example.com MX
example.com SOA
mail.example.com A
ns1.example.com A
nohost.example.com ARun it for 60 seconds with 20 client sockets and no rate limit. Do this on a lab server or a quiet window only, because an unthrottled run takes every spare CPU cycle the server has.
dnsperf -s "${HOST_IP}" -d /tmp/queries-auth.txt -l 60 -c 20BIND answered about 203,000 queries per second with no losses in our final run. Treat that as a ceiling for this VM only, because dnsperf ran on the same 2 vCPUs as named. Across four runs on this VM size the result ranged from 146,000 (with a 300 QPS background load running) to 203,000. This excerpt is the statistics block:
Queries sent: 12167421
Queries completed: 12167421 (100.00%)
Queries lost: 0 (0.00%)
Response codes: NOERROR 10139518 (83.33%), NXDOMAIN 2027903 (16.67%)
Average packet size: request 32, response 63
Run time (s): 60.000263
Queries per second: 202789.461106
Average Latency (s): 0.000394 (min 0.000008, max 0.006376)promtool query instant runs PromQL from the shell, which is quicker than the web UI when you’re checking a rule. These four are the queries we’d keep at hand: queries per second by record type, probe latency, the SERVFAIL share and the zone serial. The last one is in double quotes so the shell fills in your zone:
promtool query instant http://localhost:9090 'sum by (type) (rate(bind_incoming_queries_total[5m]))'
promtool query instant http://localhost:9090 'probe_duration_seconds{job=~"dns_.*"}'
promtool query instant http://localhost:9090 'sum(rate(bind_response_rcodes_total{rcode="SERVFAIL"}[5m])) / sum(rate(bind_response_rcodes_total[5m]))'
promtool query instant http://localhost:9090 "bind_zone_serial{zone_name=\"${DNS_ZONE}\"}"Read the first result with its window in mind. A 5-minute rate spreads the 60-second burst across the whole window, so the per-type figures sit far below dnsperf’s per-second number. Use a 1-minute window while a test is running, and the 5-minute one for alerts, where smoothing is what you want:
{type="A"} => 29690.85013904762 @[1792325889.466]
{type="SOA"} => 7422.736996476189 @[1792325889.466]
{type="MX"} => 7296.150140166667 @[1792325889.466]
probe_duration_seconds{instance="10.0.1.50:53", job="dns_soa"} => 0.000352775 @[1792325889.482]
probe_duration_seconds{instance="10.0.1.50:53", job="dns_resolve"} => 0.000494383 @[1792325889.482]
{} => 0 @[1792325889.498]
bind_zone_serial{instance="localhost:9119", job="bind", view="_default", zone_name="example.com"} => 2026101801 @[1792325889.514]The SERVFAIL share was 0 after the -4 fix. On a busy resolver, that ratio is the number you graph first, because upstream trouble and DNSSEC failures show up there before anyone files a ticket. Also note that bind_exporter’s metric names differ from older guides. There is no bind_resolver_cache_hits in the current exporter with the default stats groups, so cache behaviour comes from bind_resolver_cache_rrsets and the resolver query counters instead.
Import the BIND dashboard in Grafana
Grafana comes from its own APT repository. TLS, the reverse proxy and hardening are covered in our Grafana install guide with Nginx and SSL, so this section only does the minimum to get a dashboard on screen. Add the repository and install the OSS package:
sudo apt install -y apt-transport-https wget gnupg
sudo mkdir -p /etc/apt/keyrings
sudo wget -q -O /etc/apt/keyrings/grafana.asc https://apt.grafana.com/gpg-full.key
sudo chmod 644 /etc/apt/keyrings/grafana.asc
echo "deb [signed-by=/etc/apt/keyrings/grafana.asc] https://apt.grafana.com stable main" | sudo tee /etc/apt/sources.list.d/grafana.list
sudo apt update
sudo apt install -y grafanaRather than clicking through the data source form, provision Prometheus from a file. A fixed uid keeps dashboards that reference it stable across rebuilds:
sudo vim /etc/grafana/provisioning/datasources/prometheus.yamlAdd the data source. The editable: false line stops someone changing it in the UI, so the file stays the single place where the URL is defined.
apiVersion: 1
datasources:
- name: Prometheus
type: prometheus
uid: prometheus
access: proxy
url: http://localhost:9090
isDefault: true
editable: falseBefore you start it, know what you are exposing: Prometheus on 9090 has no authentication, Grafana on 3000 starts with admin/admin, and both listen on every interface, so restrict both ports to your admin network with your firewall (we did not test firewall rules in this lab). Start Grafana. The first start runs about 720 database migrations, which took 29 to 37 seconds in our runs, so the web server does not answer straight away:
sudo chown root:grafana /etc/grafana/provisioning/datasources/prometheus.yaml
sudo systemctl daemon-reload
sudo systemctl enable --now grafana-serverAfter about a minute, ask the API whether the provisioned data source can reach Prometheus. The default admin/admin login still works at this point because nobody has logged in yet:
curl -s -u admin:admin http://localhost:3000/api/datasources/uid/prometheus/health“Successfully queried the Prometheus API” is the answer you want. A connection refused just means Grafana is still migrating:
{"details":{"application":"Prometheus","features":{"rulerApiEnabled":false}},"message":"Successfully queried the Prometheus API.","status":"OK"}Log in at http://10.0.1.50:3000 as admin/admin and set a new password when Grafana asks. Then open http://10.0.1.50:3000/dashboard/import, type 12309 in the Grafana.com field and click Load. The bind_exporter README links to this dashboard, “Bind9 Exporter DNS”, last updated in July 2024. It has no data source input, so it picks up the default Prometheus data source on its own:
Click Import. The top of the dashboard covers restarts, CPU, file descriptors, memory and the query counters. The two stat tiles show how long ago named started and last reloaded its config, so a value that keeps resetting means something is restarting BIND behind your back.
We ran every panel’s query against our Prometheus, because a dashboard that renders is not the same as a dashboard that is right. Before node_exporter was installed, three of the 18 panels came back empty. With it, all 18 showed data, but three of them do not measure what their titles suggest, and you would only notice by comparing the numbers with ps:
- Named CPU Time, File Descriptors and Memory query
process_*on the bind job - Those series belong to the bind_exporter process, not to named
- named’s real figures are the
bind_process_*series
The difference is not small. The Memory panel showed 22.1 MiB resident, while bind_process_resident_memory_bytes reported 53.7 MiB for named, and ps agreed with both (22,676 KB for the exporter, 55,020 KB for named). If you rely on those panels, edit their queries to the bind_process_ names. The three host panels without titles stay empty until node_exporter runs, which is why we installed it.
Scroll down for the traffic. The 60-second dnsperf burst stands out in Response Results, and the resolver row shows the recursive lookups from the probe and a small background load we left running:
The dashboard has no panel for the blackbox probes. Add one with probe_success{job=~"dns_.*"} and another with probe_duration_seconds, or reuse the panels from our blackbox exporter uptime monitoring guide, which uses the same metrics for HTTP checks.
Alert when BIND stops answering
Dashboards tell you what happened. Alerts tell you while it’s happening, so the last piece is a rules file. These four rules cover a dead exporter, a dead statistics channel, a failed client-side probe and a rising SERVFAIL share:
sudo vim /etc/prometheus/rules/dns.ymlAdd the rules. BindExporterDown is there because bind_up == 0 can only fire while the exporter is alive to report it: if bind_exporter itself dies, the series just stops, and only Prometheus’s own up metric notices. The for: 1m on the first three means a single failed scrape or probe never pages anyone, while a real outage still alerts in about a minute.
groups:
- name: dns
rules:
- alert: BindExporterDown
expr: up{job="bind"} == 0
for: 1m
labels:
severity: critical
annotations:
summary: "bind_exporter on {{ $labels.instance }} is not being scraped"
- alert: BindDown
expr: bind_up == 0
for: 1m
labels:
severity: critical
annotations:
summary: "BIND on {{ $labels.instance }} stopped answering the statistics channel"
- alert: DnsProbeFailed
expr: probe_success{job=~"dns_.*"} == 0
for: 1m
labels:
severity: critical
annotations:
summary: "{{ $labels.job }} probe against {{ $labels.instance }} is failing"
- alert: DnsServfailRatioHigh
expr: |
sum by (instance) (rate(bind_response_rcodes_total{rcode="SERVFAIL"}[5m]))
/ sum by (instance) (rate(bind_response_rcodes_total[5m])) > 0.05
for: 5m
labels:
severity: warning
annotations:
summary: "More than 5% of answers from {{ $labels.instance }} are SERVFAIL"Check the file with promtool, which also catches PromQL typos, then reload Prometheus. A rule with a typo would otherwise load as an error on the Rules page and never fire, which is worse than no rule at all.
promtool check config /etc/prometheus/prometheus.yml
sudo systemctl reload prometheuspromtool now reports the rule file along with the main config. If there is no “rule files found” line at all, the glob in rule_files matched nothing, which usually means the file has a .yaml extension instead of .yml.
Checking /etc/prometheus/prometheus.yml
SUCCESS: 1 rule files found
SUCCESS: /etc/prometheus/prometheus.yml is valid prometheus config file syntax
Checking /etc/prometheus/rules/dns.yml
SUCCESS: 4 rules foundThen break it on purpose, because an alert you have never seen fire is an alert you can’t trust. Stop named and watch http://10.0.1.50:9090/alerts in your browser.
sudo systemctl stop namedWe polled the alerts API every 3 seconds during the drill:
- +3 s: the dns_soa probe alert went pending
- +18 s: BindDown and the dns_resolve probe alert went pending too
- +64 s: the dns_soa alert fired
- +79 s: all three were firing
- +18 s after
sudo systemctl start named: no active alerts
Expect some spread around those numbers, because the for timer only starts at the first rule evaluation that sees a failed scrape, and a 15 second scrape plus a 15 second evaluation interval can add up to 30 seconds of delay. Across three drills on this setup, all three alerts were firing between 67 and 82 seconds after the stop. Plan for about 90 seconds from outage to alert with these settings.
The firing state looks like this. Note that the BindDown alert carries instance="localhost:9119", the exporter’s address, while the probe alerts carry the DNS server’s address:
Bring named back with sudo systemctl start named, then test the exporter rule the same way by stopping bind_exporter itself:
sudo systemctl stop prometheus-bind-exporterBindExporterDown went pending after 18 seconds and fired at 75. BindDown stayed silent the whole time, because with the exporter gone there is no bind_up series left to compare, which is exactly the gap the extra rule covers. The alert cleared 27 seconds after the restart. Start it again with sudo systemctl start prometheus-bind-exporter.
The SERVFAIL rule did not fire in our lab because the ratio stayed at 0 after the IPv4 fix, so its 5% threshold is a starting point, not a tested value. Prometheus only evaluates rules; to get the alerts into email, Slack or a pager, route them through Prometheus Alertmanager, which we did not configure here.
Run it across more than one DNS server
One host is fine for learning. In production you have a primary, secondaries and probably a couple of resolvers, and the setup above changes in a few predictable places:
- bind_exporter on every BIND server, statistics channel still on 127.0.0.1
- Prometheus scraping each exporter on 9119 over the private network
- blackbox probes from a host outside the DNS servers, listed in
allow-recursion - One probe target per server, so a single dead secondary stands out
probe_dns_serialcompared across servers to catch stalled zone transfers
Relabel bind_exporter’s instance to the DNS server’s name, because localhost:9119 in an alert tells you nothing when there are six of them. Also open 9119 only to the Prometheus server, since the exporter has no authentication unless you give it a --web.config.file. Recursion needs one more change. BIND’s default allow-recursion covers only localnets and localhost, so a blackbox prober on another network gets REFUSED for dns_resolve, which fails the NOERROR check and fires the alert on a healthy server. Add the prober’s address to allow-recursion on each resolver, or run the recursive probe only from inside the networks those resolvers serve. If you’re comparing resolvers before you roll this out, our BIND vs dnsmasq vs PowerDNS vs Unbound comparison covers the trade-offs, and the same Prometheus server can watch your web tier too, as the Apache monitoring guide shows with apache_exporter.
