After a Checkpoint VSX firewall was upgraded from R80.30 to R81.10, the interface monitoring (using check_nwc_health plugin), stopped working on the vs0 (physical) interfaces.
Besides a major change in the SNMPv3 authentication protocol - which requires a workaround to re-enable the SHA1 SNMP protocol in Checkpoint Gaia OS - there was also a change in the way the interfaces are presented by SNMP.
Since the upgrade, the interfaces were not recognized anymore and monitoring on these interfaces stopped working:
$ ./check_nwc_health --hostname CheckPointFW --protocol 3 --username snmpuser --authpassword secret --authprotocol sha --mode interface-usage --name "eth1-02"
UNKNOWN - no interfaces
Note: The interface checks in virtual firewalls (vsN) using SNMPv3 context naming still worked!
However when logged in on the firewall (by SSH), the interface does clearly exist:
[Expert@vsx:0]# ip a sh eth1-02
9: eth1-02: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc noqueue master bond0 state UP qlen 1000
link/ether 00:1c:7f:6c:6b:2b brd ff:ff:ff:ff:ff:ff
An overview of the available interfaces can be retrieved by using --mode list-interfaces:
$ ./check_nwc_health --hostname CheckPointFW --protocol 3 --username snmpuser --authpassword secret --authprotocol sha --mode list-interfaces
000001 lo
000002 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 2
000003 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 3
000004 Intel Corporation I350 Gigabit Network Connection 4
000005 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 5
000006 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 6
000007 Intel Corporation I350 Gigabit Network Connection 7
000008 Intel Corporation I350 Gigabit Network Connection 8
000009 Intel Corporation I350 Gigabit Network Connection 9
000010 Intel Corporation I350 Gigabit Network Connection 10
000011 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 11
000012 Intel Corporation I350 Gigabit Network Connection 12
000013 Intel Corporation I350 Gigabit Network Connection 13
000014 Intel Corporation I350 Gigabit Network Connection 14
000015 Intel Corporation I350 Gigabit Network Connection 15
000016 Intel Corporation I350 Gigabit Network Connection 16
000017 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 17
000018 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 18
000019 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 19
000020 gre0
000021 gretap0
000040 bond0
000041 bond1
OK - have fun
Clearly the naming creates confusion. Instead of the actual interface name used by the Linux Kernel, the physical interface type is shown.
As the monitoring worked in the previous version (R80.30), the question pops up, how these interfaces were represented in SNMP before. Research led to issue #254 in the check_nwc_health repository, where the same interface name change was already reported after an upgrade to R80.40. The reason for this is documented in CheckPoint SK168601:
The output on R80.40 and higher versions has changed and polling ifDesc provides descriptive information of the interface (Such as driver).
This can be verified with check_nwc_health, by appending multiple verbose layers:
$ ./check_nwc_health --hostname CheckPointFW --protocol 3 --username snmpuser --authpassword secret --authprotocol sha --mode list-interfaces -vvv
000001 lo
000002 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 2
000003 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 3
000004 Intel Corporation I350 Gigabit Network Connection 4
000005 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 5
000006 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 6
000007 Intel Corporation I350 Gigabit Network Connection 7
000008 Intel Corporation I350 Gigabit Network Connection 8
000009 Intel Corporation I350 Gigabit Network Connection 9
000010 Intel Corporation I350 Gigabit Network Connection 10
000011 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 11
000012 Intel Corporation I350 Gigabit Network Connection 12
000013 Intel Corporation I350 Gigabit Network Connection 13
000014 Intel Corporation I350 Gigabit Network Connection 14
000015 Intel Corporation I350 Gigabit Network Connection 15
000016 Intel Corporation I350 Gigabit Network Connection 16
000017 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 17
000018 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 18
000019 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ 19
000020 gre0
000021 gretap0
000040 bond0
000041 bond1
[INTERFACESUBSYSTEM]
bootTime: 1642470476.81
duplicates: HASH(0x557a5f5b1f20)
ifCacheLastChange: 1642771167
ifTableLastChange: 1642470476.81
interface_cache: HASH(0x557a5fd8ddf0)
info: checking interfaces
[INTERFACE_10]
ifAlias: ________
ifDescr: Intel Corporation I350 Gigabit Network Connection 10
ifIndex: 10
ifName: eth1-03
iflabel: Intel Corporation I350 Gigabit Network Connection 10
[...]
Both ifAlias and IfDescr fields don't contain the interface name anymore. However the ifName field shows the actual interface name as seen in the OS.
Whether this change makes sense or not is questionable. But we need to find a fix for this problem, so the physical interfaces can be monitored again.
The advantage of open source software is that many minds can work on a software - and find a solution. In this case Github user Christian Mock created pull request #276 for check_nwc_health. This code change adds a new parameter --iflabel to the monitoring plugin. By using this parameter, the plugin is told to use a different field (or order of fields) than the default (ifDescr) for interface detection and checks.
After the PR was manually applied to the current version (10.0.0.2) of check_nwc_health, the --iflabel parameter can be used. Here we use the new parameter to use the interface name found in the ifName SNMP field:
$ ./check_nwc_health --hostname CheckPointFW --protocol 3 --username snmpuser --authpassword secret --authprotocol sha --mode interface-usage --name "eth1-02" --iflabel ifName
OK - interface eth1-02 usage is in:0.00% (21002.50bit/s) out:0.04% (420297.94bit/s) | 'eth1-02_usage_in'=0.00%;;;0;100 'eth1-02_usage_out'=0.04%;;;0;100 'eth1-02_traffic_in'=21002.50;;;0;1000000000 'eth1-02_traffic_out'=420297.94;;;0;1000000000
Hurray! The check works again!
As of this day, the PR has not yet been merged into the master branch. The new parameter is therefore not yet officially released.
No comments yet.
AWS Android Ansible Apache Apple Atlassian BSD Backup Bash Bluecoat CMS Chef Cloud Coding Consul Containers CouchDB DB DNS Database Databases Docker ELK Elasticsearch Filebeat FreeBSD Galera Git GlusterFS Grafana Graphics HAProxy HTML Hacks Hardware Icinga Influx Internet Java KVM Kibana Kodi Kubernetes LVM LXC Linux Logstash Mac Macintosh Mail MariaDB Minio MongoDB Monitoring Multimedia MySQL NFS Nagios Network Nginx OSSEC OTRS Office PGSQL PHP Perl Personal PostgreSQL Postgres PowerDNS Proxmox Proxy Python Rancher Rant Redis Roundcube SSL Samba Seafile Security Shell SmartOS Solaris Surveillance Systemd TLS Tomcat Ubuntu Unix VMWare VMware Varnish Virtualization Windows Wireless Wordpress Wyse ZFS Zoneminder