Nagiosgraph insert.pl: illegal attempt to update (minimum one second step)

Written by - 0 comments

Published on - Listed in Nagios Icinga Monitoring


On a new Icinga 2 monitoring server with Nagiosgraph, I became aware of a huge nagiosgraph.log file. A short look into the log file revealed hundreds and hundreds of such errors:

Mon Nov 23 12:29:02 2015 insert.pl 21629 error RRDs::update ERR /var/nagiosgraph/rrd/myhost/HTTP___size.rrd_min: illegal attempt to update using time 1448278091 when last update time is 1448278131 (minimum one second step)
Mon Nov 23 12:29:02 2015 insert.pl 21629 error ds = [
  '/var/nagiosgraph/rrd/myhost/HTTP___size.rrd_min',
  '1448278091:838:0'
];

The error message itself explains the problem pretty well: The RRD file cannot be updated because the data with the current timestamp (1448278091) is older than the existing last entry (1448278131). But how can this happen?

Actually I built this problem on my own. When you take a look at my article How to integrate Nagiosgraph in Icinga 2, you see that I created a wrapper-script (run as a cronjob) which searches for the perfdata logs and combines them. As my search command I used the following find command:

find $perfdatadir/ -type f -regex ".*[0-9]"
/var/spool/icinga2/nagiosgraph/nagiosgraph-host-perfdata.1448278031
/var/spool/icinga2/nagiosgraph/nagiosgraph-host-perfdata.1448278046
/var/spool/icinga2/nagiosgraph/nagiosgraph-perfdata.1448278046
/var/spool/icinga2/nagiosgraph/nagiosgraph-perfdata.1448278031

The script then goes through all the perfdata log files (in the order found by the find command) and combines them. As you can see above, the perfdata file with timestamp 1448278046 was found BEFORE the file with timestamp 1448278031. So the content of the newer file is parsed BEFORE the content of the older file. Hence the error message from Nagiosgraph.

I modified my find command to sort the found files by age:

find $perfdatadir/ -type f -regex ".*[0-9]" -exec ls -tr {} +
/var/spool/icinga2/nagiosgraph/nagiosgraph-perfdata.1448278811
/var/spool/icinga2/nagiosgraph/nagiosgraph-host-perfdata.1448278811
/var/spool/icinga2/nagiosgraph/nagiosgraph-perfdata.1448278826
/var/spool/icinga2/nagiosgraph/nagiosgraph-host-perfdata.1448278826
/var/spool/icinga2/nagiosgraph/nagiosgraph-perfdata.1448278841
/var/spool/icinga2/nagiosgraph/nagiosgraph-host-perfdata.1448278841

And now the errors are gone.


Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.

RSS feed

Blog Tags:

  AWS   Android   Ansible   Apache   Apple   Atlassian   BSD   Backup   Bash   Bluecoat   CMS   Chef   Cloud   Coding   Consul   Containers   CouchDB   DB   DNS   Database   Databases   Docker   ELK   Elasticsearch   Filebeat   FreeBSD   Galera   Git   GlusterFS   Grafana   Graphics   HAProxy   HTML   Hacks   Hardware   Icinga   Influx   Internet   Java   KVM   Kibana   Kodi   Kubernetes   LVM   LXC   Linux   Logstash   Mac   Macintosh   Mail   MariaDB   Minio   MongoDB   Monitoring   Multimedia   MySQL   NFS   Nagios   Network   Nginx   OSSEC   OTRS   Office   PGSQL   PHP   Perl   Personal   PostgreSQL   Postgres   PowerDNS   Proxmox   Proxy   Python   Rancher   Rant   Redis   Roundcube   SSL   Samba   Seafile   Security   Shell   SmartOS   Solaris   Surveillance   Systemd   TLS   Tomcat   Ubuntu   Unix   VMWare   VMware   Varnish   Virtualization   Windows   Wireless   Wordpress   Wyse   ZFS   Zoneminder