In this guide we are going to check how to monitor linux servers with Grafana and prometheus.
Grafana is a graphing tool, it takes data and displays them in a visually appealing maner that is easy to interpret and analyze. Prometheus is a metric server, it collects metric data periodically from a specified host such as a running linux server.
Grafana
Grafana is an opensource visualization tool. It allows you to visualize tools using dashboards that take information from server monitoring tool like Prometheus then display this information.
Prometheus
Prometheus is an open source server and application metrics tool. it can also be configured to alert.
Grafana and prometheus are widely adopted and are open source thus popular for monitoring.
Installing Node Exporter
The Prometheus Node Exporter exposes a wide variety of hardware- and kernel-related metrics.
node_exporter
is an official package that should be installed on Linux servers to be monitored. It exposes multiple hardware and OS metrics, which will be pulled by Prometheus and eventually visualized on Grafana.
Get the latest release from the release page here https://github.com/prometheus/node_exporter/releases
curl -LO https://github.com/prometheus/node_exporter/releases/download/v1.2.0/node_exporter-1.2.0.linux-amd64.tar.gz
tar -xzvf node_exporter-1.2.0.linux-amd64.tar.gz
sudo mv node_exporter-1.2.0.linux-amd64/node_exporter /usr/local/bin/
Create a systemd file /etc/systemd/system/node_exporter.service
to manage the service
# Create prometheus user
sudo useradd -s /sbin/false prometheus
cat > /etc/systemd/system/node_exporter.service <<EOF
[Unit]
Description=node_exporter
Wants=network-online.target
After=network-online.target
[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/node_exporter
[Install]
WantedBy=multi-user.target
EOF
Start Node Exporter
sudo systemctl enable --now node_exporter
sudo systemctl status node_exporter
Node Exporter metrics
Once the Node Exporter is installed and running, you can verify that metrics are being exported by cURLing the /metrics endpoint:
curl http://localhost:9100/metrics
Adding Server to Prometheus
Your locally running Prometheus instance needs to be properly configured in order to access Node Exporter metrics. The following prometheus.yml example configuration file will tell the Prometheus instance to scrape, and how frequently, from the Node Exporter via localhost:9100:
global:
scrape_interval: 15s
scrape_configs:
- job_name: node_exporter
static_configs:
- targets: ['localhost:9100']
Create a prometheus.yml
config file with jobs
cat > /etc/prometheus/prometheus.yml <<EOF
- job_name: 'node_exporter'
basic_auth:
username: 132668
password: eyJrIjoiODYyMmFiNTVjNzc1ZmJhODlkOWIzNWRjYjRjZWVmZTY1NjIxZWUyOCIsIm4iOiJtYWluLXB1Ymxpc2gta2V5IiwiaWQiOjUwNzY3OX0=
static_configs:
- targets: ['https://prometheus-blocks-prod-us-central1.grafana.net/api/prom/push']
EOF
sudo chown -R prometheus:prometheus /etc/prometheus
Use this node exporter dashboard
https://grafana.com/grafana/dashboards/1860