Modify InfluxDB subscription to send data to multiple subscribers (asynchronous data replication)

Written by - 0 comments

Published on - Listed in Influx Database


To replicate data from an InfluxDB server to another server, one of the possibilities (in the open source/community edition) is to configure a "subscription". Another blog article describes how to create an InfluxDB asynchronous replication using subscription service.

Multiple subscribers

The current subscriptions can be shown on the master server:

root@influxmaster:~# influx -username admin -password secret
Connected to http://localhost:8086 version 1.6.4
InfluxDB shell version: 1.6.4
> SHOW SUBSCRIPTIONS
name: icinga
retention_policy name               mode destinations
---------------- ----               ---- ------------
autogen          icinga-replication ALL  [http://icinga:pass@influx02:8086]

name: mtr
retention_policy name            mode destinations
---------------- ----            ---- ------------
autogen          mtr-replication ALL  [http://mtr:pass@influx02:8086]

But what if data needs to be replicated across multiple nodes (subscribers)? 

The CREATE SUBSCRIPTION command syntax allows to define multiple subscribers as last input, separated with a comma:

CREATE SUBSCRIPTION "sub0" ON "mydb"."autogen" DESTINATIONS ALL|ANY 'http://influx02:8086', 'http://influx03:8086'

Note: Very important to focus on the keywords "DESTINATIONS ALL" or "DESTINATIONS ANY". The ALL keyword stands for data replication to all defined subscribers. ANY stands for round-robin data replication across all defined subscribers!

The problem is that there is no "edit" function available for the subscriptions. Only the SHOW, CREATE and DROP commands are available. This means, the existing replication needs to be removed and then re-created with the additional subscriber.

Drop and re-create subscription

From the SHOW SUBSCRIPTIONS command we know the replication name "icinga-replication" and the source database "icinga". This subscription can now be dropped and then re-created with the additional subscriber:

> DROP SUBSCRIPTION "icinga-replication" ON "icinga"."autogen"
> CREATE SUBSCRIPTION "icinga-replication" ON "icinga"."autogen" DESTINATIONS ALL 'http://icinga:pass@influx02:8086', 'http://icinga:pass@influx03:8086'

The new subscriber should show up:

> SHOW SUBSCRIPTIONS
name: icinga
retention_policy name               mode destinations
---------------- ----               ---- ------------
autogen          icinga-replication ALL  [http://icinga:pass@influx02:8086 http://icinga:pass@influx03:8086]

Data loss during subscription re-creation

If your primary InfluxDB server continuously receives a lot of data, the data received during the DROP and CREATE SUBSCRIPTION commands will be lost. The subscription replication is a fire-and-forget kind of replication. When a subscriber does not respond, the data is not queued and then resent at a later time. The subscription service doesn't "catch up" to a certain moment in time either, such as a classical Master-Slave replication (e.g. MySQL).

To make sure all the InfluxDB servers hold the same data, stop all InfluxDB servers and manually sync /var/lib/influxdb from the master server to the others, then start InfluxDB and the subscription service replicated new incoming data.

Caution: No subscription replication in InfluxDB 2.0!

Unfortunately InfluxDB 2.0 does not offer subscription replication anymore. From the InfluxDB 1.x to 2.0 notes:

InfluxDB 2.0 has no subscriptions API and does not support Kapacitor stream tasks. To continue using stream tasks, write data directly to both InfluxDB and Kapacitor. Use Telegraf and its InfluxDB output plugin to write to Kapacitor and the InfluxDB v2 output plugin to write to InfluxDB v2.



Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.

RSS feed

Blog Tags:

  AWS   Android   Ansible   Apache   Apple   Atlassian   BSD   Backup   Bash   Bluecoat   CMS   Chef   Cloud   Coding   Consul   Containers   CouchDB   DB   DNS   Database   Databases   Docker   ELK   Elasticsearch   Filebeat   FreeBSD   Galera   Git   GlusterFS   Grafana   Graphics   HAProxy   HTML   Hacks   Hardware   Icinga   Influx   Internet   Java   KVM   Kibana   Kodi   Kubernetes   LVM   LXC   Linux   Logstash   Mac   Macintosh   Mail   MariaDB   Minio   MongoDB   Monitoring   Multimedia   MySQL   NFS   Nagios   Network   Nginx   OSSEC   OTRS   Office   PGSQL   PHP   Perl   Personal   PostgreSQL   Postgres   PowerDNS   Proxmox   Proxy   Python   Rancher   Rant   Redis   Roundcube   SSL   Samba   Seafile   Security   Shell   SmartOS   Solaris   Surveillance   Systemd   TLS   Tomcat   Ubuntu   Unix   VMWare   VMware   Varnish   Virtualization   Windows   Wireless   Wordpress   Wyse   ZFS   Zoneminder