How to ship docker logs to Grafana Loki using Docker plugin

Grafana Loki officially supports a Docker plugin that will read logs from Docker containers and ship them to Loki. 

This guide assumes that you have grafana loki up and running. If not, checkout these guides:

Installing the Docker Driver

The Docker plugin must be installed on each Docker host that will be running containers you want to collect logs from.

Run the following command to install the plugin:

docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions

To check installed plugins, use the docker plugin ls command. Plugins that have started successfully are listed as enabled:

$ docker plugin ls
ID             NAME          DESCRIPTION           ENABLED
5572cb555698   loki:latest   Loki Logging Driver   true

Configuring the Docker Driver

The Docker daemon on each machine has a default logging driver and each container will use the default driver unless configured otherwise.

The docker run command can be configured to use a different logging driver than the Docker daemon’s default with the --log-driver flag. Any options that the logging driver supports can be set using the --log-opt <NAME>=<VALUE> flag. --log-opt can be passed multiple times for each option to be set.

The following command will start Postgres in a container and send logs to local loki installation, using a batch size of 400 entries and no more than 5 retries if a send fails.

docker run --log-driver=loki \
    --log-opt loki-url="http://localhost:3100/loki/api/v1/push" \
    --log-opt loki-retries=5 \
    --log-opt loki-batch-size=400 \
    postgres:latest

The same can be achieved in docker-compose using the following:

version: '3.9'

services:
  app:
    image: postgres:latest
    restart: always
    ports:
      - 5432:5432
    logging:
      driver: loki
      options:
        loki-url: "http://localhost:3100/loki/api/v1/push"

Change the default logging driver

If you want the Loki logging driver to be the default for all containers, change Docker’s daemon.json file (located in /etc/docker on Linux) and set the value of log-driver to loki:

{
  "debug": true,
  "log-driver": "loki"
}

Options for the logging driver can also be configured with log-opts in the daemon.json:

{
    "debug" : true,
    "log-driver": "loki",
    "log-opts": {
        "loki-url": "http://localhost:3100/loki/api/v1/push",
        "loki-batch-size": "400"
    }
}

After changing daemon.json, restart the Docker daemon for the changes to take effect. All newly created containers from that host will then send logs to Loki via the driver.

Configure the logging driver for a Swarm service or Compose

You can also configure the logging driver for a swarm service directly in your compose file. This also applies for docker-compose:

version: "3.7"
services:
  logger:
    image: grafana/grafana
    logging:
      driver: loki
      options:
        loki-url: "http://localhost:3100/loki/api/v1/push"

You can then deploy your stack using:

docker stack deploy my_stack_name --compose-file docker-compose.yaml

Upgrading

The upgrade process involves disabling the existing plugin, upgrading, then re-enabling and restarting Docker:

docker plugin disable loki --force
docker plugin upgrade loki grafana/loki-docker-driver:latest --grant-all-permissions
docker plugin enable loki
systemctl restart docker

Uninstalling

To cleanly uninstall the plugin, disable and remove it:

docker plugin disable loki --force
docker plugin rm loki
Last updated on Oct 14, 2024 11:46 +0300
comments powered by Disqus
Citizix Ltd
Built with Hugo
Theme Stack designed by Jimmy