Nagiosgraph insert.pl: illegal attempt to update (minimum one second step)

Written by Claudio Kuenzler - 0 comments

Published on November 23rd 2015 - Listed in Nagios Icinga Monitoring

On a new Icinga 2 monitoring server with Nagiosgraph, I became aware of a huge nagiosgraph.log file. A short look into the log file revealed hundreds and hundreds of such errors:

Mon Nov 23 12:29:02 2015 insert.pl 21629 error RRDs::update ERR /var/nagiosgraph/rrd/myhost/HTTP___size.rrd_min: illegal attempt to update using time 1448278091 when last update time is 1448278131 (minimum one second step)
Mon Nov 23 12:29:02 2015 insert.pl 21629 error ds = [
  '/var/nagiosgraph/rrd/myhost/HTTP___size.rrd_min',
  '1448278091:838:0'
];

The error message itself explains the problem pretty well: The RRD file cannot be updated because the data with the current timestamp (1448278091) is older than the existing last entry (1448278131). But how can this happen?

Actually I built this problem on my own. When you take a look at my article How to integrate Nagiosgraph in Icinga 2, you see that I created a wrapper-script (run as a cronjob) which searches for the perfdata logs and combines them. As my search command I used the following find command:

find $perfdatadir/ -type f -regex ".*[0-9]"
/var/spool/icinga2/nagiosgraph/nagiosgraph-host-perfdata.1448278031
/var/spool/icinga2/nagiosgraph/nagiosgraph-host-perfdata.1448278046
/var/spool/icinga2/nagiosgraph/nagiosgraph-perfdata.1448278046
/var/spool/icinga2/nagiosgraph/nagiosgraph-perfdata.1448278031

The script then goes through all the perfdata log files (in the order found by the find command) and combines them. As you can see above, the perfdata file with timestamp 1448278046 was found BEFORE the file with timestamp 1448278031. So the content of the newer file is parsed BEFORE the content of the older file. Hence the error message from Nagiosgraph.

I modified my find command to sort the found files by age:

find $perfdatadir/ -type f -regex ".*[0-9]" -exec ls -tr {} +
/var/spool/icinga2/nagiosgraph/nagiosgraph-perfdata.1448278811
/var/spool/icinga2/nagiosgraph/nagiosgraph-host-perfdata.1448278811
/var/spool/icinga2/nagiosgraph/nagiosgraph-perfdata.1448278826
/var/spool/icinga2/nagiosgraph/nagiosgraph-host-perfdata.1448278826
/var/spool/icinga2/nagiosgraph/nagiosgraph-perfdata.1448278841
/var/spool/icinga2/nagiosgraph/nagiosgraph-host-perfdata.1448278841

And now the errors are gone.

Add a comment

Comments (newest first)

No comments yet.