How to install collectd and send metrics to Graphite/Statsd

Collectd is a daemon which collects system and application performance metrics periodically and provides mechanisms to store the values in a variety of ways, in our case sending to Graphite. Collectd gathers metrics from various sources, e.g. the operating system, applications, log files and external devices, and stores this information or makes it available over the network. Those statistics can be used to monitor systems, find performance bottlenecks (i.e. performance analysis) and predict future system load (i.e. capacity planning). Or if you just want pretty graphs of your private server and are fed up with some homegrown solution you’re at the right place, too.

In this tutorial we will explore the steps for installing and configuring Collectd to ship data to Graphite.

# Table of Content

Installing Collectd
Starting and enabling Collectd
Configuring Collectd
Configuring Apache

# 1. Installing Collectd

Collectd is available in default repositories for most common linux distributions. Install it with this command:

sudo apt install collectd

# 2. Starting and enabling the service

To start the service, use this command:

sudo systemctl start collectd

Check the status with this command:

<meta charset="utf-8">$ sudo systemctl status collectd
● collectd.service - Statistics collection and monitoring daemon
     Loaded: loaded (/lib/systemd/system/collectd.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-01-20 17:11:55 UTC; 6min ago
       Docs: man:collectd(1)
             man:collectd.conf(5)
             https://collectd.org
    Process: 27352 ExecStartPre=/usr/sbin/collectd -t (code=exited, status=0/SUCCESS)
   Main PID: 27353 (collectd)
      Tasks: 12 (limit: 4626)
     Memory: 23.5M
        CPU: 614ms
     CGroup: /system.slice/collectd.service
             └─27353 /usr/sbin/collectd

Jan 20 17:11:55 dev-debiansrv.inv.re collectd[27353]: plugin_load: plugin "irq" successfully loaded.
Jan 20 17:11:55 dev-debiansrv.inv.re collectd[27353]: plugin_load: plugin "load" successfully loaded.
Jan 20 17:11:55 dev-debiansrv.inv.re collectd[27353]: plugin_load: plugin "memory" successfully loaded.
Jan 20 17:11:55 dev-debiansrv.inv.re collectd[27353]: plugin_load: plugin "processes" successfully loaded.
Jan 20 17:11:55 dev-debiansrv.inv.re collectd[27353]: plugin_load: plugin "rrdtool" successfully loaded.
Jan 20 17:11:55 dev-debiansrv.inv.re collectd[27353]: plugin_load: plugin "swap" successfully loaded.
Jan 20 17:11:55 dev-debiansrv.inv.re collectd[27353]: plugin_load: plugin "users" successfully loaded.
Jan 20 17:11:55 dev-debiansrv.inv.re collectd[27353]: Systemd detected, trying to signal readiness.
Jan 20 17:11:55 dev-debiansrv.inv.re systemd[1]: Started Statistics collection and monitoring daemon.
Jan 20 17:11:55 dev-debiansrv.inv.re collectd[27353]: Initialization complete, entering read-loop.

To enable on boot:

sudo systectl enable collectd

# 3. Configure Collectd

The config file can be found in this path /etc/collectd/collectd.conf. Open with your editor

sudo vim /etc/collectd/collectd.conf

Update your hostname

Hostname "ubuntusrv"

The Interval parameter defines the time that collectd waits before querying data. Uncomment that line, and change the value from 10 to the shortest interval value chosen in Graphite configuration.

Interval 20

The matching of these two values is fundamental. If they do not match some data could be lost!

Collectd uses plugins to define the service that it will gather information about. Uncomment the ones you want. Ensure you uncomment the write_graphite plugin.

To monitor apache, uncomment the apache plugin.

LoadPlugin write_graphite
LoadPlugin apache

Further down in the file, there are plugins configuration blocks. Uncomment the Apache block and add an **Instance **sub-block; now the block will look like this:

<Plugin apache> 
#       <Instance "foo"> 
#               URL "http://localhost/server-status?auto" 
#               User "www-user" 
#               Password "secret" 
#               VerifyPeer false 
#               VerifyHost false 
#               CACert "/etc/ssl/ca.crt" 
#               Server "apache" 
#       </Instance> 
# 
#       <Instance "bar"> 
#               URL "http://some.domain.tld/status?auto" 
#               Host "some.domain.tld" 
#               Server "lighttpd" 
#       </Instance> 
 
        <Instance "Graphite"> 
                URL "http://mydomain/server-status?auto" 
                Server "apache" 
        </Instance> 
</Plugin>

For the write_graphite section:

<Plugin write_graphite> 
        <Node "graphite"> 
                Host "10.2.11.9" 
                Port "2003" 
                Protocol "tcp" 
                LogSendErrors true 
                # Prefix "collectd" 
                Postfix "collectd" 
                StoreRates true 
                AlwaysAppendDS false 
                EscapeCharacter "_" 
        </Node> 
</Plugin>

Remember that “2003” is the port number on which Carbon will listen.

Save and exit.

# 4. Configuring Apache

Configure the Apache virtual hosts file enabled for Graphite:

sudo vim /etc/apache2/sites-available/apache2-graphite.conf

Below the **content **block, add the following lines:

<Location "/server-stats">
        SetHandler server-status
        Require all granted
    </Location>

This means that Apache will serve statistics on the /server-stats page.

Save, exit and reload Apache:

sudo systemctl reload apache2

Check that everything is well configured by accessing http://localhost/server-stats with a web browser.

# Wrapping up

That is it! Collectd should be up and running, recording all the server activities and flushing these records to Graphite. It’s possible to configure collectd with additional plugins, or modify the configuration files to accomplish new/other tasks.