Monitoring plugin check_es_system 1.11.0 released: Show read only index(es) in output and jq is the new default JSON parser

Written by Claudio Kuenzler - 0 comments

Published on November 25th 2020 - Listed in Elasticsearch Monitoring ELK

A new version of check_es_system, a monitoring plugin to monitor Elasticsearch clusters and nodes, is available!

Version 1.11.0 is an enhancement release (requested in issue #30). The readonly check type now shows discovered index(es) marked as read_only or read_only_allow_delete when using jq as json parser in the background:

./check_es_system.sh -H escluster.example.com -P 9243 -S -u user -p password -t readonly
ES SYSTEM CRITICAL - 1 index(es) found read-only claudiotest - 116 index(es) found read-only (allow delete) logstash-2020.09.21 logstash-2020.09.04 filebeat-2020.09.04 .monitoring-es-6-2020.09.21 idx filebeat-2020.09.08 filebeat-2020.09.07 filebeat-2020.09.01 claudiotest2 .management-beats logstash-2020.09.05 filebeat-2020.09.10 logstash-2020.09.06 .triggered_watches-6 logstash-2020.09.02 filebeat-2020.09.21 logstash-2020.09.03 .monitoring-kibana-6-2020.09.21 filebeat-2020.09.03 logstash-2020.09.01 logstash-2020.09.08 kibana_sample_data_logs .kibana_7 .kibana_task_manager filebeat-2020.09.05 .security-6 .watches filebeat-2020.09.11 filebeat-2020.09.09 .kibana-6 filebeat-2020.09.02 logstash-2020.09.07 filebeat-2020.09.06

As you can see from the output, read_only and read_only_allow_delete indexes are shown differently.

Note: In case you are unaware, when Elasticsearch runs into problems (e.g. disk full), it marks the index(es) as read_only_allow_delete and new data cannot be written into these indexes anymore. See article Elasticsearch ignored disk watermark settings and enforced read only index for more information. It is therefore a very handy check to quickly identify if your Elasticsearch went into read only mode.

To be able to parse all the read only indexes from Elasticsearch's output (in json), a special function of jq is needed. From the source code:

roindexes=$(echo $settings | jq -r '.[].settings.index |select(.blocks.read_only == "true").provided_name')

The select function is used as a filter and is looking for all the keys read_only with a value of true. If it finds such a key, the name of this relevant index (provided_name) is displayed. 

Unfortunately the other supported JSON parser, jshon, does not offer such a filter or search function. Therefore the output of the discovered read only index(es) only show up when using jq as parser.

Due to this fact, the default JSON parser from this version on is jq. Both parsers remain (as for now) supported.

Add a comment

Comments (newest first)

No comments yet.