Time Series DB¶
Time Series1 databases are all the rage right now. They were explicitly designed to store data sequentially, the same way time passes. For this playground, I’ve chosen InfluxDB. Prometheus3 is also very popular, and you could use the PostgreSQL extensions4 for time series as well.
Influx has its own dialect of SQL. Make sure you look at the docs2.
Setup¶
There isn’t much to configure when you set it up. I only modified the section for graphite (uncommenting if its already there, or pasting it in). Everything else is default.
1 2 3 4 5 6 7 | [[graphite]]
enabled = true
database = "graphite"
retention-policy = ""
bind-address = ":2003"
protocol = "tcp"
consistency-level = "one"
|
The whole config (with comments stripped out) looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | root@monitoring:~ # grep -v '^#' /usr/local/etc/influxd.conf | grep -v ' #' | grep -v '^$'
[meta]
dir = "/var/db/influxdb/meta"
[data]
dir = "/var/db/influxdb/data"
wal-dir = "/var/db/influxdb/wal"
series-id-set-cache-size = 100
[coordinator]
[retention]
[shard-precreation]
[monitor]
[http]
[logging]
[subscriber]
[[graphite]]
enabled = true
database = "graphite"
retention-policy = ""
bind-address = ":2003"
protocol = "tcp"
consistency-level = "one"
[[collectd]]
[[opentsdb]]
[[udp]]
[continuous_queries]
[tls]
root@monitoring:~ #
|
When we start it up
1 2 3 | root@monitoring:~ # sysrc influxd_enable=YES
root@monitoring:~ # service influxd start
root@monitoring:~ #
|
and then check what is listening, we can see
1 2 3 4 5 6 | root@monitoring:~ # sockstat -46l
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
influxd influxd 66684 6 tcp4 127.0.0.1:8088 *:*
influxd influxd 66684 7 tcp46 *:8086 *:*
influxd influxd 66684 8 tcp46 *:2003 *:*
root@monitoring:~ #
|
We can see that in addition to listening on the default 8088, and 8086, its also listening on 2003, as we configured above.
Creating a DB¶
1 2 3 4 | $ influx
Connected to http://localhost:8086 version 1.8.0
InfluxDB shell version: 1.8.0
>
|
1 | CREATE DATABASE graphite
|
Hopefully this does the obvious. We now have a database created called graphite.
Creating a retention policy¶
What if you want to keep the data longer than the default of something like 168 hours? This is where retention policies come in to play. You can also assign whether the policy is the default for the database or not.
1 2 3 4 5 6 7 8 9 10 11 | > show retention policies on graphite
name duration shardGroupDuration replicaN default
---- -------- ------------------ -------- -------
autogen 0s 168h0m0s 1 true
> create retention policy "1year" on graphite duration 52w replication 1 default
> show retention policies on graphite
name duration shardGroupDuration replicaN default
---- -------- ------------------ -------- -------
autogen 0s 168h0m0s 1 false
1year 8736h0m0s 168h0m0s 1 true
>
|
Now your data will automatically expire after 52wks.
Conclusion¶
OK, we have a database setup that we can use. What next? We have 2 more things to do (not an ordered list).
- Setup an agent that will collect information and put it in the database
- Setup grafana so we can visualize the information we’re collecting
Footnotes and References¶
So what do you think? Leave your comments below.