A new version of check_rancher2, a monitoring plugin for Kubernetes clusters managed by SUSE Rancher, is now available! Version 1.13.0 introduces a new check type "api-token" to monitor the API token expiry used by the plugin.
In past Rancher releases (up to Rancher 2.8), API tokens could be created without an expire date (see "automatically expire" in the screenshot below.
In more recent Rancher 2 releases, the API tokens now always expire. The "Never" option has disappeared from the UI.
By default, new Rancher 2 API tokens expire after 3 months. This default is taken from the Global Setting "auth-token-max-ttl-minutes", which has a default value of 129600 (= 90 days).
When the API token, used for Rancher 2 monitoring by check_rancher2.sh, has expired, the monitoring plugin is now longer able to communicate with the Rancher 2 API and alerts with a WARNING:
I've personally ran several times into the problem with an expired monitoring API token in the past few months. Although the fix only takes a couple of minutes, it's still annoying. During that time, the workloads and cluster(s) are not monitored anymore.
The solution for this problem is to pro-actively monitor the API token's expiry. Similar to a Rancher internal certificate, the token can be seen in the API. One of the json fields show the expire date of the token.
The newly added check type "api-token" uses this information and in combination with the newly added parameter --expiry-warn alerts in advance:
$ ./check_rancher2.sh -H rancher2.example.com -U token-xxxxx -P "uwooyi6UthaaK5Boo5sieYo7Gavaizei3ieKahmohQua8Ju9mie5eeh" -S -t api-token --expiry-warn 14
CHECK_RANCHER2 OK - API Token (Monitoring) still valid (will expire in 310 days)
$ ./check_rancher2.sh -H rancher2.example.com -U token-xxxxx -P
"uwooyi6UthaaK5Boo5sieYo7Gavaizei3ieKahmohQua8Ju9mie5eeh" -S -t
api-token --expiry-warn 350
CHECK_RANCHER2 WARNING - API Token (Monitoring) will expire in 310 days
Please note that API tokens from other users cannot be monitored.
The newly added "api-token" check works in a similar way to the "local-certs" check and calculates the expiry on a date found in the API response.
Because of this, starting with version 1.13.0, the parameter --cert-warn is now deprecated and the newly added --expiry-warn parameter will be used instead.
For now, --cert-warn still works, but will internally set the value for the new expiry-warn variable.
No comments yet.
AWS Android Ansible Apache Apple Atlassian BSD Backup Bash Bluecoat CMS Chef Cloud Coding Consul Containers CouchDB DB DNS Database Databases Docker ELK Elasticsearch Filebeat FreeBSD Galera Git GlusterFS Grafana Graphics HAProxy HTML Hacks Hardware Icinga Influx Internet Java KVM Kibana Kodi Kubernetes LVM LXC Linux Logstash Mac Macintosh Mail MariaDB Minio MongoDB Monitoring Multimedia MySQL NFS Nagios Network Nginx OSSEC OTRS Observability Office OpenSearch PGSQL PHP Perl Personal PostgreSQL Postgres PowerDNS Proxmox Proxy Python Rancher Rant Redis Roundcube SSL Samba Seafile Security Shell SmartOS Solaris Surveillance Systemd TLS Tomcat Ubuntu Unix VMWare VMware Varnish Virtualization Windows Wireless Wordpress Wyse ZFS Zoneminder