Icinga 2: Advanced usage of arrays/dictionaries for monitoring of partitions

Written by - 2 comments

Published on - last updated on November 22nd 2021 - Listed in Nagios Icinga Monitoring


In the previous post (Using arrays in Icinga 2 custom attributes to monitor partitions) I wrote about how arrays can be used in custom attributes which are then used by apply rules.

Now I went a step further and created dictionaries (different sub-values in each array) in the custom attributes (now called custom variables). The goal in this case was to quickly define new warning and critical thresholds for each partition - if wanted.

To summarize the first post, the host object contained an array "vars.partitions" as a custom attribute which defined the partitions to be monitored:

object Host "linux-host" {
  import "generic-host"
  address = "192.168.1.45"
  vars.os = "Linux"
  # Define partitions of this host:
  vars.partitions = [ "/", "/var", "/srv" ]

}

An apply rule then read the array and applied a service object for each value of this array (therefore four different partition checks):

apply Service "Diskspace " for (partition in host.vars.partitions) {
  import "generic-service"
  check_command = "nrpe"
  vars.nrpe_command = "check_disk"
  vars.nrpe_arguments = [ "15%", "5%", partition ]

  assign where host.address && host.vars.os == "Linux"
}

Great so far. But there is one minor issue: All of these partitions are now using the fixed NRPE arguments 15% and 5% for the warning and critical thresholds. Wouldn't it be nice (if we were older... lalala) to define on the fly thresholds which overwrite the defaults? Yes it would...

Dictionaries (and a lot of try'n'err) to the rescue!

By rewriting the vars.partition array with values for each partition, additional parameters can be set:

object Host "linux-host" {
  import "generic-host"
  address = "192.168.1.45"
  vars.os = "Linux"
  # Define partitions of this host:
  vars.partitions.slash = { mountpoint = "/" }
  vars.partitions.slashtmp = { mountpoint = "/tmp" }
  vars.partitions.slashsrv = { mountpoint = "/srv", warn = "95%", crit = "90%" }
  vars.partitions.slashvar = { mountpoint = "/var" }

}

For each partition I created a custom attribute "vars.partitions.partitionname" which itself contains several variables.
The mountpoint variable is mandatory and as you can see only one partition (/srv) defines special warning and critical thresholds.


Note: If you ask why I added the word slash in every attribute name, just try it with "vars.partitions.var" and you will find out like me, that Icinga really doesn't like the word var appearing there...


[Update October 12th 2015: Please check out the comment of Powelleb below which takes my apply rule one step further (no need to create the slashXXX names!).]

For Icinga itself the "vars.partitions" attribute is still somewhat of an array, so I am still able to use one and the same apply rule for all partitions found in this array:

apply Service "Apply-Diskchecks" for (partition_name => config in host.vars.partitions) {
  import "generic-service"

  vars += config
  if (!vars.warn) { vars.warn = "15%" }
  if (!vars.crit) { vars.crit = "5%" }

  display_name = "Diskspace " + vars.mountpoint
  check_command = "nrpe"
  vars.nrpe_command = "check_disk"
  vars.nrpe_arguments = [ vars.warn, vars.crit, vars.mountpoint ]

  assign where host.address && host.vars.os == "Linux"
  ignore where host.vars.applyignore.partitions == true
}

OK, now it gets more complicated but hear me out, it's worth it!

First the definition of the apply rule which I named "Apply-Diskchecks" for each partition found in the array "host.vars.partitions"  (as defined in the Host object above).
The word "config" here seems to be a fixed alias in the dictionary usage which stands for the main value (or the array name); in this case slash, slashtmp, slashsrv and slashvar.
Now to something important: Each variable (vars) now uses the config as a prefix. Ergo if I use vars.mountpoint now it is in reality vars.partitions.slash.mountpoint if we're in the loop for the slash partition.
Right after that I define the default thresholds, if the thresholds were not set within the dictionary.
The display_name is a combination of the string "Diskspace" followed by the value of vars.mountpoint.
In vars.nrpe_arguments I am now submitting the dynamically created values of vars.warn, vars.crit and vars.mountpoint. Which means: If I didn't use special thresholds, the ones just defined a few lines above are applied. vars.mountpoint is a mandatory variable (as I wrote above), therefore this is coming from the custom attributes of the host object itself.

The thresholds and actual NRPE arguments can be checked and verified in the Icinga Classic UI (in newer Icingaweb2 as well):

Icinga 2: Custom Variables Partition
Icinga 2 Custom Attributes used for partition checks

As you see, the /srv partition contains the special thresholds defined in the host object. Success!

The NRPE check command on the server to be monitored looks like this by the way:

command[check_disk]=/usr/lib/nagios/plugins/check_disk -w $ARG1$ -c $ARG2$ -W $ARG1$ -K $ARG2$ -p $ARG3$


Add a comment

Show form to leave a comment

Comments (newest first)

ck from St. Gallen, Switzerland wrote on Oct 12th, 2015:

Hi Powelleb, nice catch, thanks!


Powelleb from USA wrote on Oct 10th, 2015:

great information. i took your post and came up with this solution to simplify needing to create names after the partitions....

object Host "linux-host" {
import "generic-host"
address = "192.168.1.45"
vars.os = "Linux"
# Define partitions of this host:
vars.partitions["/"] = {}
vars.partitions["/tmp"]= {}
vars.partitions["/srv"] = {warn = "95%", crit = "90%" }
vars.partitions["/var"] = {}
}

note the partition_name use....
apply Service "Apply-Diskchecks" for (partition_name => config in host.vars.partitions) {
import "generic-service"

vars += config
if (!vars.warn) { vars.warn = "15%" }
if (!vars.crit) { vars.crit = "5%" }

display_name = "Diskspace " + partition_name
check_command = "nrpe"
vars.nrpe_command = "check_disk"
vars.nrpe_arguments = [ vars.warn, vars.crit, partition_name ]

assign where host.address && host.vars.os == "Linux"
ignore where host.vars.applyignore.partitions == true
}


displayname and servicename same

apply Service "Diskspace " for (partition_name => config in host.vars.partitions) {
import "generic-service"

vars += config
if (!vars.warn) { vars.warn = "15%" }
if (!vars.crit) { vars.crit = "5%" }

check_command = "nrpe"
vars.nrpe_command = "check_disk"
vars.nrpe_arguments = [ vars.warn, vars.crit, partition_name ]

assign where host.address && host.vars.os == "Linux"
ignore where host.vars.applyignore.partitions == true
}






RSS feed

Blog Tags:

  AWS   Android   Ansible   Apache   Apple   Atlassian   BSD   Backup   Bash   Bluecoat   CMS   Chef   Cloud   Coding   Consul   Containers   CouchDB   DB   DNS   Database   Databases   Docker   ELK   Elasticsearch   Filebeat   FreeBSD   Galera   Git   GlusterFS   Grafana   Graphics   HAProxy   HTML   Hacks   Hardware   Icinga   Influx   Internet   Java   KVM   Kibana   Kodi   Kubernetes   LVM   LXC   Linux   Logstash   Mac   Macintosh   Mail   MariaDB   Minio   MongoDB   Monitoring   Multimedia   MySQL   NFS   Nagios   Network   Nginx   OSSEC   OTRS   Office   PGSQL   PHP   Perl   Personal   PostgreSQL   Postgres   PowerDNS   Proxmox   Proxy   Python   Rancher   Rant   Redis   Roundcube   SSL   Samba   Seafile   Security   Shell   SmartOS   Solaris   Surveillance   Systemd   TLS   Tomcat   Ubuntu   Unix   VMWare   VMware   Varnish   Virtualization   Windows   Wireless   Wordpress   Wyse   ZFS   Zoneminder