Install Grafana on Ubuntu 26.04 LTS

When you install Grafana on Ubuntu 26.04, the Prometheus that apt hands you for its data is not the current one. Grafana is the easy half, since Grafana Labs runs its own APT repository and new builds land there the day they ship. The Ubuntu archive still carries Prometheus 2.x without upstream’s web UI, so the backend your dashboards read from deserves a real decision, not a single apt line.

Original content from computingforgeeks.com - post 166133

We rebuilt the whole stack on a fresh Ubuntu 26.04 VM in November 2026 with Grafana 13.3.0 from apt.grafana.com, Prometheus 2.53.5 from the archive and 3.15.1 upstream, and every screen below comes from that run. You get the first login and the visual refresh preview, both Prometheus paths, the Node Exporter dashboard, email alerts, Nginx with HTTPS, SQLite against PostgreSQL, and an upgrade that catches the [plugins_marketplace] rename. For the Prometheus side alone, see the Prometheus guide for Ubuntu 26.04.

Prerequisites

Grafana is light at idle but not small on disk. The package we pulled was a 397 MB download that unpacked to 1.4 GB under /usr/share/grafana, and the service sat at about 52 MB of RAM right after its first start. The host needs the following.

• Ubuntu 26.04 LTS server: with a sudo user. Our Ubuntu 26.04 initial server setup covers that part.
• 2 vCPU, 4 GB RAM, 20 GB disk: this is what the lab VM had, and it’s a floor. What grows the box is the number of people loading dashboards at once, the panel count per dashboard and how much history Prometheus keeps on the same disk.
• A DNS name: only for the HTTPS section. We use grafana.example.com as the example throughout.
• Ports: 3000/tcp while you test directly, then 80 and 443 once Nginx fronts Grafana.

For a team of ten or more with busy dashboards, 4 vCPU and 8 GB is a sensible starting point, and move Prometheus to its own machine before it starts competing with Grafana for memory.

Install Grafana on Ubuntu 26.04 from the official repository

Ubuntu’s own archive does not carry a grafana package at all on 26.04. Running apt-cache madison grafana on our VM listed only apt.grafana.com rows, so the vendor repository is the only apt route. Start with the tools the key download needs.

sudo apt-get install -y apt-transport-https wget gnupg

Then put the signing key in /etc/apt/keyrings and add the stable repository. These are the steps from the official Debian and Ubuntu install page, which uses the gpg-full.key ASCII key rather than the older dearmored file:

sudo mkdir -p /etc/apt/keyrings
sudo wget -q -O /etc/apt/keyrings/grafana.asc https://apt.grafana.com/gpg-full.key
sudo chmod 644 /etc/apt/keyrings/grafana.asc
echo "deb [signed-by=/etc/apt/keyrings/grafana.asc] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
sudo apt-get update

Check that apt now sees the package before you install anything. If the Candidate line reads (none), the repository line or the key is wrong:

apt-cache policy grafana | head -4

On our run the candidate was the current stable release, served from the repository you just added.

grafana:
  Installed: (none)
  Candidate: 13.3.0
  Version table:

The repository also has a grafana-enterprise package, which Grafana Labs now recommends as the default because it is free to run and behaves like OSS until you add a licence. We install the OSS build, grafana, which is what most readers searching for this want:

sudo apt-get install -y grafana

The package does not start the service on its own, and the postinst script says so in the last lines of the apt output.

Setting up grafana (13.3.0) ...
### NOT starting on installation, please execute the following statements to configure grafana to start automatically using systemd
 sudo /bin/systemctl daemon-reload
 sudo /bin/systemctl enable grafana-server
### You can start grafana-server by executing
 sudo /bin/systemctl start grafana-server

Reload systemd so it picks up the new unit file, then enable and start the service in one go. This is the same order the postinst message recommends, and running both lines together means the service also comes back after the next reboot without a separate enable step.

sudo systemctl daemon-reload
sudo systemctl enable --now grafana-server

Three checks tell you it worked: the unit is active, grafana server -v prints the build, and something is listening on 3000. Give it about half a minute before the port check. On our VM the listener opened 26 seconds after systemd started the unit, once the plugin loader had finished:

systemctl status grafana-server --no-pager | head -9
grafana server -v
sudo ss -tlnp | grep 3000

This is the output from our VM, with the unit running and port 3000 open on all interfaces, which is the default bind address until the Nginx section narrows it.

Use grafana server -v rather than grafana-server -v. The old binary name still works, but it now prints a deprecation warning before the version, which breaks scripts that parse the first line.

Open the firewall for the first login

UFW is installed but inactive on a fresh 26.04 server. If you turn it on over SSH, allow OpenSSH first, or ufw enable cuts your own session and you’ll need the console to get back in:

sudo ufw allow OpenSSH
sudo ufw allow 3000/tcp comment "Grafana"
sudo ufw --force enable
sudo ufw status numbered

The rule list should show SSH and 3000 for both IPv4 and IPv6, because UFW adds the v6 twin of every rule on its own.

Status: active

     To                         Action      From
     --                         ------      ----
[ 1] OpenSSH                    ALLOW IN    Anywhere
[ 2] 3000/tcp                   ALLOW IN    Anywhere                   # Grafana
[ 3] OpenSSH (v6)               ALLOW IN    Anywhere (v6)
[ 4] 3000/tcp (v6)              ALLOW IN    Anywhere (v6)              # Grafana

Port 3000 stays open only until the Nginx section, where Grafana moves behind the proxy and the rule is deleted.

What happens on the first login

Browse to http://10.0.1.50:3000 (your server’s address) and sign in as admin with the password admin. Grafana does not open the home page yet, because it notices the default password and sends you to a password form first:

1. New password: type the password that replaces admin.
2. Confirm new password: the same value again.
3. Submit: saves it and opens the home page.
4. Skip: leaves admin/admin in place, which is fine on a throwaway VM and a bad idea on anything your network can reach.

Once you submit, you land on the home page in the default theme. The orange menu accents and blue primary buttons are what the default Grafana theme looks like unless you switch on the refresh in the next section:

Several commands later in this guide call the HTTP API, so keep the new credentials in a shell variable for the session instead of retyping them. Swap in the password you just set:

export GF_AUTH='admin:your-new-password'

If you ever lose that password, sudo grafana cli admin reset-admin-password 'NewPassword' on the server resets it and prints Admin password changed successfully. It writes straight to the database, so it works even when the web login doesn’t.

How the new visual refresh is switched on

The current release line ships Grafana’s biggest visual change since version 7: new greys, softer borders, white primary buttons and reworked typography. It is in public preview, so it sits behind the grafana.visualDesignRefresh feature toggle and stays off unless you set it. Grafana Labs describes it as a reskin with no changes to UX flows or navigation and no breaking changes for plugins that use the standard theme, and we didn’t hit anything that worked differently with it on. Open the main config file:

sudo vim /etc/grafana/grafana.ini

Find the [feature_toggles] section (it’s uncommented in the stock file) and add the toggle on the line below the header:

[feature_toggles]
grafana.visualDesignRefresh = true

Restart Grafana and ask the API which toggles the frontend received. Read the key name in the reply, not only the value, because a typo in the key fails silently.

sudo systemctl restart grafana-server
curl -s -u "$GF_AUTH" localhost:3000/api/frontend/settings | python3 -c 'import json,sys; ft=json.load(sys.stdin).get("featureToggles",{}); print({k:v for k,v in ft.items() if "isual" in k})'

A working toggle comes back as True under its exact name. We also tried a deliberate typo, grafana.visualDesignRefersh. Grafana accepted it, the journal showed no “unknown flag” warning for it, and the API returned it as True as well, so the spelling in this output and the changed look in the browser are the only real checks.

{'grafana.visualDesignRefresh': True}

Reload the browser and the same home page now uses the refreshed theme. Compare the near-black canvas, the floating content panel and the white “Create your first dashboard” button with the earlier screenshot:

Every other screenshot in this guide was taken with the toggle on, so this is what you’ll see if you follow along with it enabled. To go back, delete the line and restart. Nothing in the database depends on it.

Which Prometheus to install: the Ubuntu package or upstream 3.x

The Ubuntu package is the low-effort choice and upstream 3.x is the current one. Which is right for you depends on whether security updates from Ubuntu matter more to you than features and the modern UI. The archive is behind for a structural reason, not because nobody noticed. Ubuntu 26.04 took Debian’s prometheus build as it stood when the release froze, and Debian rebuilds Prometheus from its own packaged Go libraries rather than upstream’s vendored ones, which makes every major bump slow. Debian has since moved to a 3.x build in unstable and testing, according to its package tracker, but a frozen LTS archive only takes security and bug fixes, so 26.04 stays on 2.x for its lifetime. The Debian build also drops upstream’s React web UI to stay inside Debian policy. Its /graph page serves a notice that “This Debian package of Prometheus does not include the modern React web UI”, and you get the classic UI instead.

	Ubuntu package	Upstream tarball
Version line	2.x	3.x
Web UI	Classic UI only	Current React UI
Updates	Ubuntu security fixes through apt	You download and swap the binary
Config and data	/etc/prometheus, /var/lib/prometheus/metrics2	Wherever your unit points (we use /var/lib/prometheus/data)
Setup effort	One apt command	About ten commands and a unit file

Grafana’s Prometheus data source talks to both the same way, so nothing on the Grafana side changes with your choice. If this is a lab or a single box you want patched without thinking about it, take the package. If you want 3.x features such as UTF-8 metric names, Remote Write 2.0 and the new UI, or you plan to follow upstream docs closely, take the tarball. Node Exporter comes from the Ubuntu archive in both paths, because its archive build is recent enough that we saw no reason to replace it.

Option A: the Ubuntu archive package

One command installs both services, and unlike Grafana, their packages enable and start them for you.

sudo apt-get install -y prometheus prometheus-node-exporter
systemctl is-active prometheus prometheus-node-exporter

Both should print active. These three commands show where the archive stands.

apt-cache policy prometheus prometheus-node-exporter | grep -E "^[a-z]|Candidate"
prometheus --version 2>&1 | head -1
prometheus-node-exporter --version 2>&1 | head -1

The server is the 2.x build with a debian/sid branch tag, which is the Debian package carried over unchanged:

The packaged /etc/prometheus/prometheus.yml already scrapes itself on 9090 and Node Exporter on 9100, so there is nothing to edit before Grafana can use it. Confirm both targets are healthy:

curl -s http://localhost:9090/api/v1/targets | python3 -c 'import json,sys; [print(t["labels"]["job"], t["scrapeUrl"], t["health"]) for t in json.load(sys.stdin)["data"]["activeTargets"]]'

Both jobs should report up within one scrape interval of the service starting. A down here usually means the exporter is not running, so check systemctl status prometheus-node-exporter before you look at Grafana. Grafana can only show what this endpoint already has.

node http://localhost:9100/metrics up
prometheus http://localhost:9090/metrics up

With Option A done you can skip straight to adding the data source. One detail matters if you change your mind later: the packaged service listens on all interfaces, so keep 9090 out of your UFW rules unless other hosts really need it.

Option B: upstream Prometheus 3.x

If you tried Option A first, remove its server package before you start. The purge also deletes /etc/prometheus and the 2.x data in /var/lib/prometheus/metrics2, so copy anything you want to keep first. Purge promtool in the same command, because the archive ships it as a separate package and it would leave an old 2.x /usr/bin/promtool behind. Node Exporter isn’t touched:

sudo systemctl disable --now prometheus
sudo apt-get purge -y prometheus promtool

On a fresh server, install Node Exporter from the archive. Its postinst also creates the prometheus system user and /var/lib/prometheus, which the upstream service below runs as and writes into:

sudo apt-get install -y prometheus-node-exporter

Download the release tarball and its checksum file, and verify the archive before you unpack it. The first line asks GitHub for the latest stable tag, which skips release candidates, and strips its leading v:

PROM_VERSION=$(curl -s https://api.github.com/repos/prometheus/prometheus/releases/latest | grep '"tag_name"' | cut -d '"' -f 4 | sed 's/^v//')
echo "$PROM_VERSION"
cd /tmp
wget -q https://github.com/prometheus/prometheus/releases/download/v${PROM_VERSION}/prometheus-${PROM_VERSION}.linux-amd64.tar.gz
wget -q https://github.com/prometheus/prometheus/releases/download/v${PROM_VERSION}/sha256sums.txt
sha256sum --check --ignore-missing sha256sums.txt
tar xzf prometheus-${PROM_VERSION}.linux-amd64.tar.gz

The echo shows the tag it resolved, and the checksum line must end in OK. Anything else means a truncated or tampered download, so fetch it again instead of continuing:

3.15.1
prometheus-3.15.1.linux-amd64.tar.gz: OK

Install the two binaries into /usr/local/bin, which keeps them clear of anything apt manages, and create the config and data directories:

cd /tmp/prometheus-${PROM_VERSION}.linux-amd64
sudo install -m 0755 prometheus promtool /usr/local/bin/
sudo mkdir -p /etc/prometheus /var/lib/prometheus/data

Create the scrape config, which tells Prometheus what to collect and how often. Two jobs are enough to start, because the Node Exporter dashboard later in this guide reads only the node job. Add more targets once the basics work, so a typo in a new job can’t hide a problem in the first two.

sudo vim /etc/prometheus/prometheus.yml

This is the exact file we ran. It scrapes Prometheus itself and the local Node Exporter every 15 seconds:

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]

  - job_name: "node"
    static_configs:
      - targets: ["localhost:9100"]

Hand the directories to the service user and let promtool check the YAML before systemd ever sees it:

sudo chown -R prometheus:prometheus /etc/prometheus /var/lib/prometheus/data
promtool check config /etc/prometheus/prometheus.yml

A valid file reports SUCCESS, and a YAML error prints the line number instead.

Checking /etc/prometheus/prometheus.yml
 SUCCESS: /etc/prometheus/prometheus.yml is valid prometheus config file syntax

Now write the systemd unit, since the tarball ships only binaries and no service file.

sudo vim /etc/systemd/system/prometheus.service

Add the unit below. The --web.listen-address=127.0.0.1:9090 flag is a deliberate choice. Grafana on the same host reaches it over loopback, and nothing else on the network can query your metrics or the unauthenticated admin endpoints:

[Unit]
Description=Prometheus monitoring system
Documentation=https://prometheus.io/docs/introduction/overview/
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/var/lib/prometheus/data \
  --storage.tsdb.retention.time=30d \
  --web.listen-address=127.0.0.1:9090
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure

[Install]
WantedBy=multi-user.target

Load it, start it, and then check the version and the listeners in one pass.

sudo systemctl daemon-reload
sudo systemctl enable --now prometheus
systemctl status prometheus --no-pager | head -3
prometheus --version | head -1
sudo ss -tlnp | grep -E "9090|9100"

Our run showed the 3.x server bound to loopback only, while Node Exporter from the archive listens on every interface:

That *:9100 is why UFW matters here: with the firewall on and no 9100 rule, the exporter is reachable from the host but not from the network. The same /api/v1/targets check from Option A returns both jobs as up on the upstream server too. To upgrade later, repeat the download and install lines and restart the unit. The TSDB carries across 3.x releases.

Add Prometheus as a data source

Grafana stores nothing itself. Each panel sends a PromQL query through the data source to Prometheus at render time, so this connection is what every dashboard below depends on. In the side menu, open the Connections section.

1. Connections: expands the menu that holds integrations and data sources.
2. Data sources: lists what’s configured, which is nothing on a fresh install.
3. Add data source: opens the catalogue. Pick Prometheus under time series databases.

The settings page has a lot of fields, and only one of them matters on a single host. Grafana and Prometheus share the machine, so the URL is loopback:

Type http://localhost:9090 into Prometheus server URL, leave authentication at No Authentication, scroll to the bottom and click Save & test. Grafana runs a real query against the API and reports the result in a banner:

“Successfully queried the Prometheus API” means the query round-tripped, not only that the port answered. For scripted or repeatable setups, the HTTP API creates the same data source in one call:

curl -s -u "$GF_AUTH" -H "Content-Type: application/json" -X POST localhost:3000/api/datasources -d '{"name":"prometheus","type":"prometheus","url":"http://localhost:9090","access":"proxy","isDefault":true}' | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d["message"], d["datasource"]["uid"])'

It prints the confirmation and the UID Grafana assigned, which dashboards and provisioning files reference later.

Datasource added bg33s899ypou8c

Pick one method. Running the API call after the UI form fails with a name conflict, because both create a source called prometheus.

Import the Node Exporter Full dashboard

Dashboard 1860 on grafana.com, Node Exporter Full, is the usual first dashboard because it already knows every metric Node Exporter exposes. Open Dashboards, then New and Import:

1. Dashboard ID 1860: type the ID into the grafana.com field. The server fetches the JSON itself, so it needs outbound HTTPS.
2. Load: pulls the latest revision and shows the import options.

On our VM, Load sat on a spinner for several seconds while it fetched from grafana.com. Older guides then tell you to pick the Prometheus data source on the options page. The current revision does not ask, because it carries a data source variable and selects your default Prometheus source on its own, so you only click Import. After half an hour of scraping, the dashboard looks like this:

The top row confirms the dashboard reads the right host: 2 CPU cores, 4 GiB RAM, 18 GiB root filesystem and no swap, which matches the lab VM exactly. The CPU spikes in the graph come from package installs we ran while the dashboard was collecting. If the gauges show N/A everywhere, the Instance drop-down isn’t pointing at localhost:9100. The same pattern works for other exporters, for example in our guides to monitor MySQL and MariaDB with Prometheus, Redis metrics in Grafana and Linux server uptime dashboards. Once you start editing the panels, our PromQL cheat sheet covers the rate and histogram queries those panels are built from, and our list of Grafana and Prometheus books goes further than the docs take you.

How alert emails get from Grafana to a mailbox

Grafana alerting sends email through whatever relay the [smtp] section of grafana.ini names, and contact points only choose the recipients. That split is why a contact point can look correct and still never deliver: the relay settings live in a file, not in the UI. In the lab we pointed Grafana at a local Postfix installed as “Local only”, which proves the path without a mail account. If you already have a relay from your provider, skip this install and use its details in the next step. Otherwise preseed the debconf answers so the install doesn’t stop at a menu:

echo "postfix postfix/main_mailer_type select Local only" | sudo debconf-set-selections
echo "postfix postfix/mailname string node-a.lab" | sudo debconf-set-selections
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y postfix mailutils
systemctl is-active postfix

It should print active, with Postfix listening on 127.0.0.1:25 only, so it delivers to local mailboxes and nothing outside the host can relay through it. Next, open the Grafana config.

sudo vim /etc/grafana/grafana.ini

The [smtp] keys ship commented out with ;. These are the values we enabled for the local relay.

[smtp]
enabled = true
host = localhost:25
from_address = [email protected]
from_name = Grafana node-a
startTLS_policy = NoStartTLS

For a real provider, point host at its submission port (for example smtp.example.com:587), uncomment user and password, and set startTLS_policy = MandatoryStartTLS, so a downgrade attack can’t strip TLS. A password containing # or ; has to be wrapped in triple quotes, because both start a comment in this file. The alerting docs list the rest in configure email for alert notifications. Restart Grafana, then open Alerting and Notification configuration in the side menu, as the breadcrumb in the screenshot shows, and add a contact point from the Contact points tab. The form also lives at /alerting/notifications/receivers/new if you’d rather go straight there:

1. Contact point name: what alert rules and notification policies refer to.
2. Addresses: one or more recipients.
3. Test: sends a sample notification through the relay before you save, then Send test notification in the dialog.

The UI says the test was sent as soon as Grafana hands the message to the relay, so only the relay log proves delivery.

sudo grep -E "status=|from=<grafana" /var/log/mail.log | tail -3

Postfix accepted the message from Grafana’s from address and delivered it about 20 ms later.

2026-11-02T16:43:56.874951+00:00 node-a postfix/qmgr[7254]: D4338B91D4: from=<[email protected]>, size=29839, nrcpt=1 (queue active)
2026-11-02T16:43:56.885376+00:00 node-a postfix/local[7583]: D4338B91D4: to=<gedion@localhost>, relay=local, delay=0.02, delays=0.01/0.01/0/0.01, dsn=2.0.0, status=sent (delivered to mailbox)

The message arrived with the subject [FIRING:1] TestAlert Grafana. A 29 KB test mail is normal, since the HTML template embeds its styling. If you’re used to triggering this test from a script, see the troubleshooting entry on the removed receivers endpoint.

Put Grafana behind Nginx with HTTPS

Grafana can terminate TLS itself, but Nginx in front gives you certbot’s automatic renewal and a single place for headers and rate limits. It also lets Grafana listen on loopback only. Install Nginx, certbot and the Nginx plugin together.

sudo apt-get install -y nginx certbot python3-certbot-nginx

Create the site file under sites-available, which is where Ubuntu’s Nginx package expects it.

sudo vim /etc/nginx/sites-available/grafana.conf

Paste this server block, with your own name in server_name. The map and the /api/live/ location carry the WebSocket upgrade for Grafana Live. Without them, panels still render but streaming updates and live alert state silently stop:

map $http_upgrade $connection_upgrade {
    default upgrade;
    '' close;
}

server {
    listen 80;
    server_name grafana.example.com;

    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://127.0.0.1:3000;
    }

    location /api/live/ {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header Host $host;
        proxy_pass http://127.0.0.1:3000;
    }
}

Enable the site and test the syntax before reloading, since a broken file takes every site on the box down at reload:

sudo ln -sf /etc/nginx/sites-available/grafana.conf /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

Next, tell Grafana its public URL and stop it listening on the network. Open /etc/grafana/grafana.ini with sudo vim again and set these three keys in the [server] section, removing the leading ; from each:

[server]
http_addr = 127.0.0.1
domain = grafana.example.com
root_url = https://grafana.example.com/

Get root_url wrong and logins redirect to the wrong host and links in alert emails point nowhere, because Grafana builds absolute URLs from it. Restart Grafana and request the certificate. The --redirect flag makes certbot add the HTTP to HTTPS redirect for you:

sudo systemctl restart grafana-server
sudo certbot --nginx -d grafana.example.com --non-interactive --agree-tos -m [email protected] --redirect

Our lab VM has no public DNS name, so we ran this exact command with two extra flags, --server and --no-verify-ssl, against Pebble, the test ACME server Let’s Encrypt publishes. The HTTP-01 challenge, the Nginx edit and the redirect are the same code path. The only difference is the issuing CA, which is also why the expiry date below is shorter than a real Let’s Encrypt certificate:

Account registered.
Requesting a certificate for grafana.example.com

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/grafana.example.com/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/grafana.example.com/privkey.pem
This certificate expires on 2026-11-08.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.

Deploying certificate
Successfully deployed certificate for grafana.example.com to /etc/nginx/sites-enabled/grafana.conf
Congratulations! You have successfully enabled HTTPS on https://grafana.example.com

If port 80 is not reachable from the internet, for example on a private network, use certbot’s DNS-01 challenge with your DNS provider’s plugin instead. Either way, check that plain HTTP redirects, HTTPS reaches Grafana, and the renewal timer is scheduled:

curl -s -o /dev/null -w "%{http_code} %{redirect_url}\n" http://grafana.example.com/
curl -s -o /dev/null -w "%{http_code} %{redirect_url}\n" https://grafana.example.com/
curl -s https://grafana.example.com/api/health
systemctl list-timers certbot.timer --no-pager | head -3

We added -k to the HTTPS calls in the lab because Pebble’s root isn’t trusted. With a Let’s Encrypt certificate you don’t need it. A 301 on HTTP and a 302 to /login on HTTPS is the healthy result:

301 https://grafana.example.com/
302 https://grafana.example.com/login
{
  "database": "ok",
  "version": "13.3.0",
  "commit": "6f8ce3c693b493043ebcd4787566d2b6b118b0f0"
}
NEXT                        LEFT LAST PASSED UNIT          ACTIVATES
Tue 2026-11-03 03:49:29 UTC  11h -         - certbot.timer certbot.service

Now swap the firewall rules. Port 3000 is only bound to loopback at this point, so its rule does nothing but add confusion:

sudo ufw allow "Nginx Full"
sudo ufw delete allow 3000/tcp
sudo ufw status

The final rule set is SSH plus HTTP and HTTPS, with nothing left that points at Grafana directly.

Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere
Nginx Full                 ALLOW       Anywhere
OpenSSH (v6)               ALLOW       Anywhere (v6)
Nginx Full (v6)            ALLOW       Anywhere (v6)             

From here on, everyone reaches Grafana at https://grafana.example.com, and the localhost:3000 API calls in this guide still work from the server itself.

What the Grafana CLI still handles

Most administration has moved into the UI and the API, but the grafana cli subcommand is still the tool for plugins and for resetting the admin password from the server. Plugin listings changed in the current line. Core data sources such as Prometheus, Loki and Tempo now install as separate plugins under /var/lib/grafana/plugins with their own version numbers:

sudo grafana cli plugins ls

Our fresh install listed 17 of them, and their version numbers do not follow the Grafana build.

installed plugins:

grafana-advisor-app @ 1.1.1
influxdb @ 13.1.10
prometheus @ 13.2.5
cloudwatch @ 12.11.2
zipkin @ 12.4.12
grafana-exploretraces-app @ 3.1.0
elasticsearch @ 12.9.4
grafana-lokiexplore-app @ 3.1.0
grafana-postgresql-datasource @ 13.0.6
opentsdb @ 13.0.7
grafana-metricsdrilldown-app @ 3.1.0
jaeger @ 13.0.7
loki @ 13.2.5
grafana-pyroscope-datasource @ 13.0.9
mssql @ 13.0.9
tempo @ 13.4.1
grafana-pyroscope-app @ 3.1.0

That has a practical consequence. The plugins directory is now part of what makes your install work, so it belongs in every backup, and plugins update-all updates the data sources too. Installing a community plugin works the same as before, followed by a restart:

sudo grafana cli plugins install grafana-clock-panel
sudo systemctl restart grafana-server

The restart is required. Grafana scans plugins only at startup, and the CLI reminds you with “Please restart Grafana after installing or removing plugins”.

Where SQLite stops being enough

Grafana keeps users, dashboards, alert rules and data source definitions in /var/lib/grafana/grafana.db, a SQLite file. It is fine for one server and a handful of editors. It struggles when many people save dashboards or alert state at the same moment, and it can’t be shared by two Grafana instances behind a load balancer. Before you jump to PostgreSQL, there’s a cheaper fix for write contention in the [database] section of grafana.ini:

[database]
wal = true

Write-ahead logging lets readers carry on while a write is in progress. After a restart you can see it working, because SQLite keeps two extra files next to the database:

sudo ls -la /var/lib/grafana/ | grep grafana.db

The -wal and -shm files are new, and they stay there while Grafana runs.

-rw-r-----  1 grafana grafana 2359296 Nov  2 16:49 grafana.db
-rw-r-----  1 grafana grafana   32768 Nov  2 16:50 grafana.db-shm
-rw-r-----  1 grafana grafana   86552 Nov  2 16:50 grafana.db-wal

Those files are also why the upgrade section stops Grafana before copying the database. A copy of grafana.db alone, taken while the service runs in WAL mode, can miss recent writes.

Switching to PostgreSQL starts an empty Grafana

For high availability or a large team, PostgreSQL is the right backend. The part most guides leave out is that changing type doesn’t move anything. Grafana runs its schema migrations into the new database and starts as if freshly installed. On our VM that was 728 migrations in about eight seconds, after which the dashboard list and data sources were empty and the old admin password returned 401. Decide on the backend before you build content, or plan to export and re-import it. Create the role and database:

sudo apt-get install -y postgresql
sudo -u postgres psql -c "CREATE ROLE grafana WITH LOGIN PASSWORD 'your-db-password';" -c "CREATE DATABASE grafana OWNER grafana;"

Both statements should echo back their command tag, which is how psql reports success.

CREATE ROLE
CREATE DATABASE

Then set the connection in the [database] section of grafana.ini. The Ubuntu 26.04 PostgreSQL package listens on localhost only by default, so ssl_mode = disable is acceptable here and wrong for a remote database:

[database]
type = postgres
host = 127.0.0.1:5432
name = grafana
user = grafana
password = your-db-password
ssl_mode = disable

Restart Grafana and count the migrations it recorded, then check which of the two passwords works now.

sudo systemctl restart grafana-server
sudo -u postgres psql -d grafana -tAc "select count(*) from migration_log;"
curl -s -o /dev/null -w "%{http_code}\n" -u "$GF_AUTH" https://grafana.example.com/api/user
curl -s -o /dev/null -w "%{http_code}\n" -u admin:admin https://grafana.example.com/api/user

The migration count, then 401 for your old password and 200 for the default one, confirms you’re on a new, empty database (we added -k for the Pebble certificate again):

728
401
200

Log in as admin/admin right away and change the password again, because the forced-change form from the first login comes back on this database too. Then run export GF_AUTH='admin:your-new-password' again with the new value, or the API checks later in this guide return 401. The switch also leaves the data source and dashboard from earlier behind in SQLite, so either rebuild them or, if you were only testing, set type back to sqlite3 and restart. That’s what we did, and the old password worked again straight away because grafana.db was never touched.

Query fan-out and log volume

Two more settings are worth changing on a shared server. The stock max_conns_per_host under [dataproxy] is 0, which means unlimited, so one 50-panel dashboard opened by ten people can open hundreds of connections to Prometheus at once. A cap queues them instead. The log level defaults to info, and log rotation already keeps seven days by default in [log.file], not in [log] where many guides put it. These are the values we ran:

[dataproxy]
max_conns_per_host = 25

[log]
level = warn

[log.file]
max_days = 14

After a restart, the journal showed no level=info lines at all in the 40 seconds we watched, only warnings. One thing you won’t find in the OSS config is query result caching: there’s no [caching] section in the defaults file, and the Cache tab on the data source page carries the Enterprise and Cloud badge.

How to upgrade Grafana in place

An upgrade is an apt install of a newer package, and for most releases that is all it is. Grafana Labs’ upgrade guide for the current line has no version-specific steps beyond backups and a plugin update. We tested the real path by installing the previous release from the same repository, giving it a customised [marketplace] block and a plugin, then upgrading. Stop Grafana first so the database is consistent, and copy the three things that make the install yours:

sudo systemctl stop grafana-server
BACKUP=/var/backups/grafana-$(date +%F)
sudo mkdir -p "$BACKUP"
sudo cp -a /etc/grafana/grafana.ini /var/lib/grafana/grafana.db /var/lib/grafana/plugins "$BACKUP"/
sudo ls -la "$BACKUP"

The backup holds the config, the database and the plugin tree with ownership preserved by cp -a. This section assumes SQLite. On PostgreSQL, take a pg_dump grafana in place of the grafana.db copy:

total 1692
drwxr-xr-x  3 root    root       4096 Nov  2 16:55 .
drwxr-xr-x  3 root    root       4096 Nov  2 16:55 ..
-rw-r-----  1 grafana grafana 1617920 Nov  2 16:55 grafana.db
-rw-r-----  1 root    grafana  100693 Nov  2 16:54 grafana.ini
drwxr-x--- 19 grafana grafana    4096 Nov  2 16:55 plugins

Upgrade only the Grafana package, so the change window does not pull in unrelated updates from the rest of the system.

sudo apt-get update
sudo apt-get install -y --only-upgrade grafana

Watch the last lines. On an upgrade, the package restarts the service itself, even though you stopped it, so Grafana is back up on the new build the moment apt returns:

Unpacking grafana (13.3.0) over (13.2.5) ...
Setting up grafana (13.3.0) ...
Restarting grafana-server service... OK

Don’t call the upgrade finished yet. Two checks remain: the config section rename and the plugins.

Why a customised [marketplace] block stops working

The GitHub release notes for the current line flag one config change as breaking: the [marketplace] section is renamed to [plugins_marketplace] (grafana/grafana#132172). The current package ships the new name in both sample.ini, which becomes /etc/grafana/grafana.ini on a fresh install, and defaults.ini, so new installs never see it. Upgrades do, because grafana.ini isn’t a dpkg conffile and the package never rewrites it. Your old section stays exactly where it was, Grafana reads it as an unknown section without logging a warning, and the setting falls back to its default. Check the file and the effective settings:

sudo grep -n -A3 "^\[marketplace\]" /etc/grafana/grafana.ini
curl -s -u "$GF_AUTH" localhost:3000/api/admin/settings | python3 -c 'import json,sys; d=json.load(sys.stdin); print("marketplace:", d.get("marketplace")); print("plugins_marketplace:", d.get("plugins_marketplace"))'

On our upgraded VM, the custom license_directory was still in the file, while the section Grafana actually reads had gone back to /var/lib/grafana:

The section only holds the directory for per-plugin marketplace licences, and the pull request notes that the features reading it sit behind a pre-GA feature toggle, so on a typical OSS install nothing breaks today. The risk is a stale setting that silently stops applying once those features ship. Environment variable users get the same rename: GF_MARKETPLACE_LICENSE_DIRECTORY becomes GF_PLUGINS_MARKETPLACE_LICENSE_DIRECTORY, which matters for Docker and Kubernetes deployments. Rename the header and restart.

sudo sed -i 's/^\[marketplace\]$/[plugins_marketplace]/' /etc/grafana/grafana.ini
sudo systemctl restart grafana-server

Run the same api/admin/settings check again. The old section is gone and the new one carries your value.

marketplace: None
plugins_marketplace: {'license_directory': '/etc/grafana/licenses'}

Finish with the plugins, which the Grafana docs recommend updating after every upgrade because a new core can break old plugin builds:

sudo grafana cli plugins update-all
sudo systemctl restart grafana-server
grafana server -v

The clock panel we pinned to an older release on the old Grafana moved to the current build, and the server reports the new version.

✔ Downloaded and extracted grafana-clock-panel v3.2.4 zip successfully to /var/lib/grafana/plugins/grafana-clock-panel

Please restart Grafana after installing or removing plugins. Refer to Grafana documentation for instructions if necessary.

Version 13.3.0 (commit: 6f8ce3c693b493043ebcd4787566d2b6b118b0f0, branch: release-13.3.0#patched)

If the new build misbehaves, the backup is your way back, and we tested this rollback on the same VM. Restore the files before the downgrade, because the package restarts Grafana as it installs and should come up on the old database, not the migrated one. Delete any leftover WAL files first, since a stale -wal replayed onto a restored database can corrupt it. The first line picks the newest backup directory, so it still works in a new shell or on another day (check the echo before you continue), and you replace the plugins directory as a whole rather than copying over it:

BACKUP=$(ls -d /var/backups/grafana-* | tail -1)
echo "$BACKUP"
sudo systemctl stop grafana-server
sudo rm -f /var/lib/grafana/grafana.db-wal /var/lib/grafana/grafana.db-shm
sudo cp -a "$BACKUP"/grafana.ini /etc/grafana/grafana.ini
sudo cp -a "$BACKUP"/grafana.db /var/lib/grafana/grafana.db
sudo rm -rf /var/lib/grafana/plugins
sudo cp -a "$BACKUP"/plugins /var/lib/grafana/plugins
sudo apt-get install -y --allow-downgrades grafana=13.2.5
grafana server -v

Put the version you came from in the grafana= pin. Ours came back on the old build with a healthy database, and the clock panel was back at its older release. On PostgreSQL, restore your pg_dump file with psql in place of the grafana.db copy, as the Grafana upgrade guide describes. Never downgrade the package without restoring the old database, because the newer build may already have migrated the schema.

Troubleshooting

Every error below came up during the lab run. The first one hits almost every freshly cloned cloud image.

E: Could not get lock /var/lib/dpkg/lock-frontend

Our first apt-get install failed with “It is held by process 1994 (apt-get)”. That process was cloud-init running a first-boot dist-upgrade, which on our clone also installed a new kernel and rebooted the VM partway through. Wait it out with cloud-init status --wait before installing anything, then reboot if /var/run/reboot-required exists. Killing the process leaves dpkg half-configured.

dial tcp [::1]:9090: connect: connection refused

We stopped Prometheus and asked Grafana to check the data source, which returned “Post “http://localhost:9090/api/v1/query”: dial tcp [::1]:9090: connect: connection refused”. The [::1] misleads people. Go resolves localhost to ::1 first and reports the failure for that address, even though the upstream service in this guide listens on 127.0.0.1. The fix is to start Prometheus (systemctl status prometheus). If you want error messages that match the 127.0.0.1 listener exactly, set the data source URL to http://127.0.0.1:9090.

FAILED: open /etc/prometheus/prometheus.yml: permission denied

promtool printed this on our first attempt, because the file had been copied in with mode 0600 and the prometheus user couldn’t read it. The service fails the same way at startup. A file created with sudo vim gets 0644 and doesn’t hit this, but a file copied from a home directory with a strict umask does. Run sudo chmod 0644 /etc/prometheus/prometheus.yml and check again.

This endpoint has been removed. Please use /apis/notifications.alerting.grafana.app/…

Scripts that tested contact points with a POST to /api/alertmanager/grafana/config/api/v1/receivers/test get this message back on the current line. The endpoint moved to the new notifications.alerting.grafana.app API group that the message names. For a one-off check, the Test button on the contact point does the same job, as shown in the email section.

plugin process exited … error=”signal: terminated” on every restart

Each systemctl restart grafana-server logs a burst of level=error msg="plugin process exited" lines, one per backend plugin, such as zipkin, cloudwatch and the PostgreSQL data source. They’re harmless. Grafana stops its plugin subprocesses on shutdown and logs their exit at error level. Only errors with a timestamp after the restart, from the new process ID, point at a real problem. sudo journalctl -u grafana-server --since "-2min" narrows the view to that window.