In this guide we will go through the steps of exporting the logs generated by apps running in a kubernetes cluster into grafana loki.
Grafana Loki is a log aggregation tool inspired by prometheus. It provides you a central place where all your logs can be shipped to. Grafana provides a UI that can be accessed over https to check and search your logs.
With logs in Loki you can configure alerting based on set conditions for your log data like when an error occurs.
Promtail is end of life. Grafana declared Promtail EOL on March 2, 2026; it gets no more updates, and all new development happens in Grafana Alloy. The
grafana/promtailandgrafana/loki-stackHelm charts used below are both marked deprecated. For a new cluster, jump to Shipping logs with Grafana Alloy. The Promtail sections are kept for clusters that still run it. To move an existing Promtail config over, see the official Promtail-to-Alloy migration guide.
Installing Loki with Helm
Add the Grafana repository to Helm and update it.
helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
Install the loki-stack with Helm:
helm install loki-stack grafana/loki-stack \
--create-namespace \
--namespace loki-stack \
--set promtail.enabled=true,loki.persistence.enabled=true,loki.persistence.size=100Gi
To install in an existing namespace
helm install loki-stack grafana/loki-stack \
--namespace <namespace> \
--set promtail.enabled=true,loki.persistence.enabled=true,loki.persistence.size=100Gi
With promtail.enabled=true, loki-stack already deploys Promtail as a DaemonSet next to Loki, so you do not need to install it separately.
The
loki-stackchart is deprecated and pinned to Loki 2.9. For a current Loki install, use thegrafana/lokichart as shown in How to Run Grafana Loki with Helm and Kustomize in Kubernetes.
Ensure that the application is running:
kubectl get pv,pvc -n loki-stack
kubectl get pods -n loki-stack
Sending logs from a standalone host
To send data using Promtail, the simplest approach is to run it with docker.
The examples below push to Grafana Cloud Logs. Replace <INSTANCE_ID> with your Loki instance ID (the user), <API_KEY> with a Grafana Cloud access policy token that has the logs:write scope, and <LOKI_HOST> with your Loki host (for example logs-prod-us-central1.grafana.net). All three are on your stack’s Loki details page. Save it to promtail/config.yaml.
server:
http_listen_port: 0
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: https://<INSTANCE_ID>:<API_KEY>@<LOKI_HOST>/loki/api/v1/push
scrape_configs:
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: varlogs
__path__: /var/log/*.log
Run promtail via docker:
docker run --name promtail --volume "$PWD/promtail:/etc/promtail" --volume "/var/log:/var/log" grafana/promtail:3.6.11 -config.file=/etc/promtail/config.yaml
To run Promtail in a Kubernetes cluster without Helm, Loki ships a script that renders a Promtail DaemonSet manifest which you can pipe into kubectl:
curl -fsS https://raw.githubusercontent.com/grafana/loki/main/tools/promtail.sh | sh -s <INSTANCE_ID> <API_KEY> <LOKI_HOST> default | kubectl apply --namespace=default -f -
Sending logs using helm
Adding Promtail DaemonSet
To ship all your pods logs, we’re going to set up Promtail as a DaemonSet in our cluster. This means it will run on each node of the cluster. We’ll then configure it to find the logs of your containers on the host.
Promtail uses the same service discovery as Prometheus
Make sure the scrape_configs of Promtail matches the Prometheus one. Not only this is simpler to configure, but this also means Metrics and Logs will have the same metadata (labels) attached by the Prometheus service discovery. Then, when querying Grafana, you’ll be able to correlate metrics and logs very quickly.
helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
helm search repo grafana/promtail
Create promtail values. Current versions of the chart take a list of Loki push endpoints under config.clients:
cat > values.yaml <<EOF
config:
clients:
- url: https://<LOKI_HOST>/loki/api/v1/push
basic_auth:
username: "<INSTANCE_ID>"
password: "<API_KEY>"
EOF
For a Loki running inside the cluster, point url at its service instead (for example http://loki-gateway.loki.svc.cluster.local/loki/api/v1/push) and drop basic_auth.
Once you’re ready let’s create a new namespace monitoring and add Promtail to it:
kubectl create namespace monitoring
helm install promtail --namespace monitoring grafana/promtail -f values.yaml
kubectl get -n monitoring pods
Shipping logs with Grafana Alloy
Grafana Alloy replaces Promtail. Its Helm chart deploys Alloy as a DaemonSet by default, and you give it the pipeline in alloy.configMap.content. The config below follows the official Kubernetes logs example: it discovers pods on each node, adds namespace, pod, container and app labels, tails the logs through the Kubernetes API, and writes them to Loki.
cat > alloy-values.yaml <<'EOF'
alloy:
configMap:
content: |
discovery.kubernetes "pod" {
role = "pod"
selectors {
role = "pod"
field = "spec.nodeName=" + coalesce(sys.env("HOSTNAME"), constants.hostname)
}
}
discovery.relabel "pod_logs" {
targets = discovery.kubernetes.pod.targets
rule {
source_labels = ["__meta_kubernetes_namespace"]
target_label = "namespace"
}
rule {
source_labels = ["__meta_kubernetes_pod_name"]
target_label = "pod"
}
rule {
source_labels = ["__meta_kubernetes_pod_container_name"]
target_label = "container"
}
rule {
source_labels = ["__meta_kubernetes_pod_label_app_kubernetes_io_name"]
target_label = "app"
}
}
loki.source.kubernetes "pod_logs" {
targets = discovery.relabel.pod_logs.output
forward_to = [loki.write.default.receiver]
}
loki.write "default" {
endpoint {
url = "https://<LOKI_HOST>/loki/api/v1/push"
basic_auth {
username = "<INSTANCE_ID>"
password = "<API_KEY>"
}
}
}
EOF
helm install alloy grafana/alloy --namespace monitoring --create-namespace -f alloy-values.yaml
kubectl get -n monitoring pods
As with Promtail, use the in-cluster Loki service URL and drop the basic_auth block when Loki runs in the same cluster. In production, keep the token in a Kubernetes Secret rather than in the values file.
Querying the logs in Grafana
You can reach your Grafana instance and start exploring your logs. For example, if you want to see all logs in the monitoring namespace use {namespace="monitoring"}. You can also expand a single log line to discover all labels available from the Kubernetes service discovery. See the LogQL documentation for filtering and aggregating queries.