How to run Prometheus with docker and docker-compose

Prometheus is a free open source software application used for event monitoring and alerting. It was originally built at SoundCloud. It is now a standalone open source project and maintained independently of any company.

Prometheus collects and stores its metrics as time series data, i.e. metrics information is stored with the timestamp at which it was recorded, alongside optional key-value pairs called labels. Metrics are numeric measurements, time series mean that changes are recorded over time. What users want to measure differs from application to application. For a web server it might be request times, for a database it might be number of active connections or number of active queries etc.

Metrics play an important role in understanding why your application is working in a certain way. If you are running a web application and find that the application is slow. You will need some information to find out what is happening with your application. For example the application can become slow when the number of requests are high. If you have the request count metric you can spot the reason and increase the number of servers to handle the load.

In this guide we will learn how to run prometheus with docker and docker-compose.

Ensure that docker and docker compose is installed

Since we will be using docker to run prometheus, it is important that it is installed and running. Please ensure that you are have docker installed. If you are using an Ubuntu system, checkout this guide on How to Install and Use Docker in Ubuntu 22.04.

Confirm that docker is working as expecting by checking the version:

$ docker version
Client: Docker Engine - Community
 Version:           20.10.17
 API version:       1.41
 Go version:        go1.17.11
 Git commit:        100c701
 Built:             Mon Jun  6 23:02:46 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.17
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.11
  Git commit:       a89b842
  Built:            Mon Jun  6 23:00:51 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.6
  GitCommit:        10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc:
  Version:          1.1.2
  GitCommit:        v1.1.2-0-ga916309
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Next, ensure that docker-compose is installed. Docker compose is available as a python pip package. Ensure that python and pip is installed then install docker-compose with this command:

sudo pip3 install docker-compose

Running prometheus with Docker

All Prometheus services are available as Docker images on Quay.io or Docker Hub.

Running Prometheus on Docker is as simple as docker run -p 9090:9090 prom/prometheus. This starts Prometheus with a sample configuration and exposes it on port 9090.

You will need some basic configuration to use. Save this as prometheus.yml in the current directory:

# my global config
global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "prometheus"

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ["localhost:9090"]

The Prometheus image uses a volume to store the actual metrics. For production deployments it is highly recommended to use a named volume to ease managing the data on Prometheus upgrades.

To provide your own configuration, there are several options. Here are two examples.

Volumes & bind-mount

Bind-mount your prometheus.yml from the host by running:

docker run \
    -p 9090:9090 \
    -v /path/to/prometheus.yml:/etc/prometheus/prometheus.yml \
    prom/prometheus:latest

Or bind-mount the directory containing prometheus.yml onto /etc/prometheus by running:

docker run \
    -p 9090:9090 \
    -v /path/to/config:/etc/prometheus \
    prom/prometheus:latest

This is the output on my system:

$ docker run \
    -p 9090:9090 \
    -v ./prometheus.yml:/etc/prometheus/prometheus.yml \
    prom/prometheus:latest
Unable to find image 'prom/prometheus:latest' locally
latest: Pulling from prom/prometheus
50e8d59317eb: Pull complete
b6c3b3e34d73: Pull complete
c25d1f04e478: Pull complete
cf87de5429d8: Pull complete
f30143b595e6: Pull complete
9d4045bcdf1f: Pull complete
11e771ad0e20: Pull complete
26449787f2fa: Pull complete
9d2e147a0f6b: Pull complete
fecd900b3277: Pull complete
e18940bc0d33: Pull complete
76c3b14215ee: Pull complete
Digest: sha256:f4c5fa1018a5c15f81250b8122f36f90146d5c2b323f9ec1f41d1b36dc0e7cb9
Status: Downloaded newer image for prom/prometheus:latest

Custom image

To avoid managing a file on the host and bind-mount it, the configuration can be baked into the image. This works well if the configuration itself is rather static and the same across all environments.

For this, create a new directory with a Prometheus configuration and a Dockerfile like this:

FROM prom/prometheus
ADD prometheus.yml /etc/prometheus/

Now build and run it:

docker build -t my-prometheus .
docker run -p 9090:9090 my-prometheus

A more advanced option is to render the configuration dynamically on start with some tooling or even have a daemon update it periodically.

Running prometheus with Docker Compose

We can add the instructions above to a docker compose file to make it easy for us to create, update and manage the deployments. Docker Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.

Add the following to a docker-compose.yaml file:

version: '3.9'

services:
  prometheus:
    image: prom/prometheus:latest
    restart: always
    ports:
      - 9090:9090
    volumes:
      - type: bind
        source: ./prometheus.yml
        target: /etc/prometheus/prometheus.yml

Save that file in the same directory where the configuration file prometheus.yml resides.

Now you can bring up the services with this command:

docker-compose up -d

This is the output in my system:

$ docker-compose up -d
Creating network "prom_default" with the default driver
Creating prom_prometheus_1 ... done

Now you can check the processes:

$ docker-compose ps
      Name                     Command               State                    Ports
-----------------------------------------------------------------------------------------------------
prom_prometheus_1   /bin/prometheus --config.f ...   Up      0.0.0.0:9090->9090/tcp,:::9090->9090/tcp

From the above we can see that the service is up and running in port 9090. To access prometheus endpoint, visit http://server_ip:9090.

Conclusion

In this guide we managed to run prometheus using docker and docker-compose.