Time series database

A time series database is a type of database that is optimized for storing data based on time stamped data. This method of storing data is useful in situations where values need to be monitored for change over time. While any database could be used to store this information, time series databases are time stamp aware which leading to improved organization and query speeds.

Time series databases have come up more in recent times due to the increase in the number of internet connected devices. Sensors can now be found throughout households, cities, and businesses. The measurements from these devices is usually precise with a frequent sample rate. A time series database is also able to aggregate this data so that large amounts of data can be removed by still keeping a summary of the samples.

The TICK stack

The TICK stack is a set of tools built around the time series database called InfluxDB. The stack is broken into its four components which each offer an option functionality.

Telegraf - A lightweight client for measuring values and sending them to the database InfluxDB - The time series database which stores the data and processes queries efficiently Chronograf - A UI to make it convenient to bring all of the other components together Kapacitor - A data monitoring and alerting tool

Setting up the TICK stack

Setup is easy by creating a docker compose file that contains each of the components as a service. This setup doesn’t contain Kapacitor because I don’t have a need for it at this time.

Telegraf

Start by creating a basic telegraf.conf file that looks like the following. This file will need to be modified again later once a token has been generated by InfluxDB. This example is only collecting CPU data. There are many other parameters that can easily be added to the configuration to have Telegraf start collecting. Examples of these can be found on the InfluxDB website or Github repo.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

[[outputs.influxdb_v2]]
 urls = ["http://influxdb:8086"]

 ## Token for authentication.
 token = "my_secret_token"

# Read metrics about cpu usage
[[inputs.cpu]]
  ## Whether to report per-cpu stats or not
  percpu = true
  ## Whether to report total system cpu stats or not
  totalcpu = true
  ## If true, collect raw CPU time metrics
  collect_cpu_time = false
  ## If true, compute and report the sum of all non-idle CPU states
  report_active = false

To run Telegraf, we’ll want to add a Telegraf service to our docker-compose.yml file.

1
2
3
4
5
6
7
8

telegraf:
  image: telegraf:latest
  restart: unless-stopped
  container_name: telegraf
  volumes:
    - ./telegraf.conf:/etc/telegraf/telegraf.conf
  depends_on:
    - influxdb

InfluxDB

With InfluxDB we create an initial configuration file in configs/influx-configs. We will need to run a command in the container and this config will be partially recreated.

1
2
3
4
5

[influx-config]
  url = "http://influxdb:8086"
  token = "my_secret_token"
  org = "example"
  active = true

Next we will add InfluxDB to the docker-compose.yml file. InfluxDB will also need a volume in order to make the data collected persist on to the host system.

1
2
3
4
5
6
7
8
9

influxdb:
  image: influxdb:2.0
  restart: unless-stopped
  container_name: influxdb
  ports:
    - "8086:8086"
  volumes:
    - ./data:/var/lib/influxdb2
    - ./config:/etc/influxdb2

We can start the InfluxDB container with docker-compose up influxdb. We then need to run through the process of generating an API key. This can be done by using the following command and walking through the prompts.

1

docker exec -it influxdb bash -c "influx config --name influx-config"

Once finished we can see the active authentications and get the API key for Telegraf and Chronograf.

1

docker exec -it influxdb bash -c "influx auth list"

Chronograf

Chronograf is the simplest of the components to setup. We simply add service to the docker compose file start it up.

1
2
3
4
5
6

chronograf:
  image: chronograf:latest
  restart: unless-stopped
  container_name: chronograf
  ports:
    - "8888:8888"

Viewing

With all of the containers running, we can visit Chronograf on port 8888 to view the data. The prompts will walk you through entering connection information and creating a dashboard. The example system dashboard will take advantage of the CPU data that was logged and show other system information fields that can could be useful to enable. The example dashboards also use the old query language called InfluxQL for their queries. They will need to be rewritten using their new language called Flux. For this reason you will not see any data in the graphs due to the old language storing data in a “database” vs the new language’s concept of “buckets”.