Monitoring plugin check_es_system 1.11.0 released: Show read only index(es) in output and jq is the new default JSON parser

Written by - 0 comments

Published on - Listed in Elasticsearch Monitoring ELK


A new version of check_es_system, a monitoring plugin to monitor Elasticsearch clusters and nodes, is available!

Version 1.11.0 is an enhancement release (requested in issue #30). The readonly check type now shows discovered index(es) marked as read_only or read_only_allow_delete when using jq as json parser in the background:

./check_es_system.sh -H escluster.example.com -P 9243 -S -u user -p password -t readonly
ES SYSTEM CRITICAL - 1 index(es) found read-only claudiotest - 116 index(es) found read-only (allow delete) logstash-2020.09.21 logstash-2020.09.04 filebeat-2020.09.04 .monitoring-es-6-2020.09.21 idx filebeat-2020.09.08 filebeat-2020.09.07 filebeat-2020.09.01 claudiotest2 .management-beats logstash-2020.09.05 filebeat-2020.09.10 logstash-2020.09.06 .triggered_watches-6 logstash-2020.09.02 filebeat-2020.09.21 logstash-2020.09.03 .monitoring-kibana-6-2020.09.21 filebeat-2020.09.03 logstash-2020.09.01 logstash-2020.09.08 kibana_sample_data_logs .kibana_7 .kibana_task_manager filebeat-2020.09.05 .security-6 .watches filebeat-2020.09.11 filebeat-2020.09.09 .kibana-6 filebeat-2020.09.02 logstash-2020.09.07 filebeat-2020.09.06

As you can see from the output, read_only and read_only_allow_delete indexes are shown differently.

Note: In case you are unaware, when Elasticsearch runs into problems (e.g. disk full), it marks the index(es) as read_only_allow_delete and new data cannot be written into these indexes anymore. See article Elasticsearch ignored disk watermark settings and enforced read only index for more information. It is therefore a very handy check to quickly identify if your Elasticsearch went into read only mode.

To be able to parse all the read only indexes from Elasticsearch's output (in json), a special function of jq is needed. From the source code:

roindexes=$(echo $settings | jq -r '.[].settings.index |select(.blocks.read_only == "true").provided_name')

The select function is used as a filter and is looking for all the keys read_only with a value of true. If it finds such a key, the name of this relevant index (provided_name) is displayed. 

Unfortunately the other supported JSON parser, jshon, does not offer such a filter or search function. Therefore the output of the discovered read only index(es) only show up when using jq as parser.

Due to this fact, the default JSON parser from this version on is jq. Both parsers remain (as for now) supported.


Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.

RSS feed

Blog Tags:

  AWS   Android   Ansible   Apache   Apple   Atlassian   BSD   Backup   Bash   Bluecoat   CMS   Chef   Cloud   Coding   Consul   Containers   CouchDB   DB   DNS   Database   Databases   Docker   ELK   Elasticsearch   Filebeat   FreeBSD   Galera   Git   GlusterFS   Grafana   Graphics   HAProxy   HTML   Hacks   Hardware   Icinga   Influx   Internet   Java   KVM   Kibana   Kodi   Kubernetes   LVM   LXC   Linux   Logstash   Mac   Macintosh   Mail   MariaDB   Minio   MongoDB   Monitoring   Multimedia   MySQL   NFS   Nagios   Network   Nginx   OSSEC   OTRS   Office   PGSQL   PHP   Perl   Personal   PostgreSQL   Postgres   PowerDNS   Proxmox   Proxy   Python   Rancher   Rant   Redis   Roundcube   SSL   Samba   Seafile   Security   Shell   SmartOS   Solaris   Surveillance   Systemd   TLS   Tomcat   Ubuntu   Unix   VMWare   VMware   Varnish   Virtualization   Windows   Wireless   Wordpress   Wyse   ZFS   Zoneminder