Header RSS Feed
 
If you only want to see the articles of a certain category, please click on the desired category below:
ALL Android Backup BSD Database Hacks Hardware Internet Linux Mail MySQL Nagios/Monitoring Network Personal PHP Proxy Shell Solaris Unix Virtualization VMware Windows Wyse

Pure-FTPd: Add full name (syntax) when creating a new virtual user
Thursday - Apr 16th 2015 - by - (0 comments)

Are you asking yourself, which argument is used to set the "full name" in pure-ftpd?
The full name is shown in in the pure-pw list command and is helpful for a description:

pure-pw list
global              /srv/www/websites/./                Global View
web1                /srv/www/websites/web1/./           web1.example.com
web2                /srv/www/websites/web2/./           web2.example.com
web3                /srv/www/websites/web3/./           web3.example.com

Of course you have checked the man page, but it only describes this:

SYNTAX
       pure-pw useradd login [-f passwd_file] [-F puredb_file] -u uid [-g gid]
                       -D/-d home_directory [-c gecos]
                       [-t download_bandwidth] [-T upload_bandwidth]
                       [-n max number_of_files] [-N max_Mbytes]
                       [-q upload_ratio] [-Q download_ratio]
                       [-r /[,/]...] [-R /[,/]...]
                       [-i /[,/]...] [-I /[,/]...]
                       [-y ]
                       [-z -] [-m]

As stupid as it sounds, but I couldn't find any document defining which parameter is used for the full name.
I only found the following hint on the official pure-ftpd documentation for Virtual Users:

Once virtual users have been created, you can edit their info. For instance
you can add bandwidth throttling, change quotas, add their full name, update
ratio, etc.

This description can cause some confusion, because you can actually add "full name" during the creation, not only after a virtual user has already been created.

After just try'n'err, I found out that the magic parameter is [-c] which was probably named for "comment", although pure-ftpd calls it "full name". So with -c the full name can be set during creation:

pure-pw useradd web4 -u ftpuser -g ftpgroup -m -d /srv/www/websites/web4/ -c "web4.example.com"

This can be verified with the pure-pw show command:

pure-pw show web4
Login              : web4
Password           : $2a$07$IIyR6Stseeq2TmQ8V8vGLu/dAs0N8Rt9z0zSpSpY5oS.7hu7y7v/6
UID                : 1000 (ftpuser)
GID                : 1001 (ftpgroup)
Directory          : /srv/www/websites/web4/./
Full name          : web4.example.com
[...]

 

Remove update uninstall folders in Windows 7 and free disk space
Saturday - Apr 11th 2015 - by - (0 comments)

For a while now, my wife complains about her laptop having no disk space anymore. I finally took the time and checked what can be done with the exception of deleting pictures and documents...

In the good ol' Windows XP days, I used to regularly delete the so-called NtUninstall subfolders in C:\Windows. They were used to be able to uninstall a previously installed Windows update.

Now in the Windows 7 (yes, 7, not 8! I still need anger management whenever I have to use a Win 8 system!) days, I was looking for these NtUninstall folders but they were gone. Thanks to an excellent article on Howtogeek, I now know that the equivalent of the NtUninstall folders in Windows 7 is the C:\Windows\winsxs folder. However this folder is not only used by the Windows updates, it is also used by Windows itself. So just randomly deleting subfolders in it is likely to cause major system issues.

Windows winsxs subfolders

However the folder is still huge as there were many Windows updates accumulated during the years which eat up disk space.

winsxs folder size 

Luckily Microsoft has back-ported a feature from Windows 8 (so there are actually good things in Win 8 it seems!) to Windows 7. Instead of manually trying to find out which subfolders could be deleted in C:\Windows\winsxs, the "Windows Update Cleanup" can be launched. This is part of the "Disk Cleanup" tool, which can be launched by right-clicking the C: drive and go into the properties:

Disk Cleanup Windows C drive Disk Cleanup for Windows C

By default, only temporary files are listed. As you can see this would only free a few hundred MB. By clicking on "Clean up system files" (requires Administrator privileges), Windows also allows to delete the previous updates:

Windows Update Cleanup Windows 7

No surprise - the Windows Update Cleanup actually allows to free most of the total amount of disk space to gain. Compared to the full folder size of C:\Windows\winsxs, the Windows Update Cleanup process is able to delete almost 40% out of the winsxs folder. 

Note: To actually see the freed disk space on the C: drive, a Windows restart was necessary.

 

Ubuntu boot bug (error: diskfilter writes are not supported)
Friday - Apr 10th 2015 - by - (0 comments)

A couple of days ago I installed a new test VM with Ubuntu 14.04.2 in my home LAN running in VMware player. An error appeared during boot:

error: diskfilter writes are not supported.
Press any key to continue...

As it was a test VM and I only needed the VM for a short time, I didn't really look that error up. But yesterday I installed a new production VM at work on ESX and hit the same error.

A quick research points me to the AskUbuntu entry #468466 and to the Ubuntu bug #1274320, a grub2 bug reported at the end of January 2014. On the AskUbuntu site, there is an excellent response why this bug happens by Rarylson Freitas.

The partition setup of this VM has the root (/) partition loaded from a logical volume, which seems to trigger this bug.

As this is a new VM, I decided to reinstall Ubuntu and to create a primary partition for the root partition. Ubuntu now boots correct without errors.

 

Installing and testing the new pywbem 0.8 (development version)
Wednesday - Apr 8th 2015 - by - (0 comments)

The python module "pywbem" is the essential piece of the monitoring plugin check_esxi_hardware. It allows to gather information from the ESXi CIM server and therefore detect hardware failures.

In the last couple of years pywbem was patched directly by several distributions (Red Hat, SuSE, Ubuntu to name a few) - yet the upstream (original) development of pywbem was stuck in version 0.7.0.

A few months ago, the pywbem project on SourceForge was revived and a new release is just about the corner. Currently there are several development releases of the new version 0.8.0 available for download. You can manually install the new version with the following steps.

1. Download the newest zip file from http://sourceforge.net/projects/pywbem/files/pywbem/pywbem-0.8.0/

2. Make sure you have python installed, otherwise install python. Simply launch the "python" command to see if python is installed:

python
Python 2.7.8 (default, Sep 30 2014, 15:34:38) [GCC] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

3. Install the python module "M2Crypto". This is a new requirement for pywbem 0.8.0. The packet name is "python-M2Crypto" in openSUSE and "python-m2crypto" in Debian/Ubuntu.

zypper in python-M2Crypto

 4. As root user, unzip the pywbem zip file and launch the python setup script:

unzip pywbem-0.8.0-dev.r728.zip
cd pywbem-0.8.0-dev

python setup.py install
[...]
running install_scripts
copying build/scripts-2.7/wbemcli.py -> /usr/bin
copying build/scripts-2.7/mof_compiler.py -> /usr/bin
changing mode of /usr/bin/wbemcli.py to 755
changing mode of /usr/bin/mof_compiler.py to 755
running install_egg_info
Writing /usr/lib/python2.7/site-packages/pywbem-0.8.0_dev-py2.7.egg-info

5. Test if pywbem can be imported as a python module:

python
Python 2.7.8 (default, Sep 30 2014, 15:34:38) [GCC] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pywbem
>>> exit()

There was no error after the import command which means the installation worked.

Now I can test the CIM connection:

python
Python 2.7.8 (default, Sep 30 2014, 15:34:38) [GCC] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pywbem
>>> wbemclient = pywbem.WBEMConnection('https://myesxi.internal', ('root', 'mypass'))
>>> c=wbemclient.EnumerateInstances('CIM_Card')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pywbem/cim_operations.py", line 1018, in EnumerateInstances
    **params)
  File "pywbem/cim_operations.py", line 592, in imethodcall
    timeout=self.timeout)
  File "pywbem/cim_http.py", line 542, in wbem_request
    h.endheaders()
  File "/usr/lib64/python2.7/httplib.py", line 991, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 844, in _send_output
    self.send(msg)
  File "pywbem/cim_http.py", line 333, in send
    self.connect()
  File "pywbem/cim_http.py", line 419, in connect
    "SSL error %s: %s" % (str(arg.__class__), arg))
pywbem.cim_http.ConnectionError: SSL error <class 'M2Crypto.SSL.SSLError'>: certificate verify failed

Oops. Error. But that's expected. One of the patches added more security in the wbem connection by validating the returned certificate. This bug is already documented in the check_esxi_hardware FAQ. As a workaround, an additional parameter can be added to the pywbem.WBEMConnection function, but in the original 0.7.0 version this parameter does not exist and therefore breaks compatibility.

But now with the newer 0.8.0 version the version name can be read:

python
Python 2.7.8 (default, Sep 30 2014, 15:34:38) [GCC] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pywbem
>>> import pkg_resources
>>> pkg_resources.get_distribution("pywbem").version
'0.8.0-dev'
>>> exit()

Hence there is now a difference now between the original 0.7.0 and the newer 0.8.0 version which can be checked before applying the additional parameter into the pywbem.WBEMConnection function.

As soon as the 0.8.0 is officially out, I hope that Linux distributions will start to roll out the upstream version again and then you can expect a final bug fix in check_esxi_hardware, too.

 

7 year old bug in rsnapshot fixed (LVM snapshot missing mount point)
Tuesday - Apr 7th 2015 - by - (0 comments)

Back in September 2014, I discovered a bug in rsnapshot (see Rsnapshot does not remove LV snapshot when mount failed). It turns out that this bug was in rsnapshot for almost 7 years - the currently available stable version is 1.3.1 which was released in September 2008. 

A few days ago, the issue I opened in Github, received a pull request which finally solves the bug.

Now when rsnapshot runs and the LVM mount point was not found, the meanwhile created LVM snapshot is correctly removed again:

# /usr/local/bin/rsnapshot alpha
[...]
/sbin/lvcreate --snapshot --size 100M --name rsnapshot \
    /dev/rsnapshotubuntu-vg/test
/bin/mount /dev/rsnapshotubuntu-vg/rsnapshot /mnt/lvm-snapshot
mount: mount point /mnt/lvm-snapshot does not exist
----------------------------------------------------------------------------
rsnapshot encountered an error! The program was invoked with these options:
/usr/local/bin/rsnapshot alpha
----------------------------------------------------------------------------
ERROR: Mount LVM snapshot failed: 8192
/usr/bin/logger -p user.err -t rsnapshot[19677] /usr/local/bin/rsnapshot \
    alpha: ERROR: Mount LVM snapshot failed: 8192
/sbin/lvremove --force /dev/rsnapshotubuntu-vg/rsnapshot
rm -f /var/run/rsnapshot.pid

Before the bug fix, the LV snapshot was taken but not removed when the mount failed (again see Rsnapshot does not remove LV snapshot when mount failed).

Kudos to Benedikt Heine, who solved the bug with his pull request!

 

VMware: Consolidate snapshots not working (file locked error)
Thursday - Apr 2nd 2015 - by - (0 comments)

I came across a VMware problem where several VM's were in a warning state with the following alert:

Virtual machine requires consolidation

In English: "Virtual machine requires consolidation".

A right-click on the "VM -> Snapshot -> Consolidate" failed with the following error:

Consolidation error - failed to lock the file

In English, the error message is "unable to access file since it is locked".
In the vmware.log file of this VM this could be followed and the virtual disk clearly failed to lock:

2015-04-02T07:01:00.608Z| vcpu-0| I120: DISK: Failed to open disk for consolidate '/vmfs/volumes/mydatastoreuuid/myvm/myvm-000009.vmdk' : Failed to lock the file (16392) 6873
2015-04-02T07:01:00.753Z| vcpu-0| I120: Vix: [173318 vigorCommands.c:548]: VigorSnapshotManagerConsolidateCallback: snapshotErr = Failed to lock the file (5:4008)

This happened on several VM's on several physical hosts. One thing was in common though: The vCenter server was recently updated and therefore there was a communication loss between vCenter server and the physical ESXi hosts.
The VMware KB entry 2003638 contains an interesting note:

    The snapshot commit fails due to locked files. For details on how to release the lock on locked files, see:
        Investigating virtual machine file locks on ESXi/ESX (10051)
        Unable to delete the virtual machine snapshot due to locked files (2017072)
    There is a temporary loss of communication between the vCenter Server and the ESXi/ESX host during snapshot commit. The most common cause of this is an environmental issue which can have a knock on affect on the ESXi/ESX host management agent hostd. For troubleshooting guidance and some possible causes of such an outage, see Troubleshooting vmware-hostd service if it fails or stops responding on an ESX/ESXi host (1002849).

Actually both hints (the lock and loss of communication) combined present the reason and the solution.
There was a loss of communication between the vCenter Server and the ESXi hosts (due to the vCenter update). Yet it is also very likely, that the backup software tried to create a snapshot during this communication loss.
The solution now is to force a re-connection of the ESXi server to the vCenter server. This can be done by restarting the hostd service:

/etc/init.d/hostd restart

After the hostd restart, wait 1 to 2 minutes and then launch "Consolidate" again.
This has worked fine on all VM's and solved the warnings.

 

check_smart: Change of syntax when -g (global) parameter is used
Thursday - Mar 12th 2015 - by - (0 comments)

As I wrote back on February 16h 2015 (check_smart: Your opinion is needed for a bugfix) there was something like an open vote on how to fix a bug on the Nagios plugin check_smart. I gave a deadline until the end of February 2015. Due to personal reasons (my Google+ contacts know why) I was not able to complete the work/fix the bug at the begin of March.

Finally I got to it and the bugfix is as suggested in issue #13: The syntax has slightly changed.

Before the bugfix, the regular expression was handled in the plugin itself, while the static input came from the user:

./check_smart.pl -g "/dev/sd" -i scsi

This would automatically check for all drives from a-z, therefore /dev/sda until /dev/sdz. However on FreeBSD systems this caused a bug because the drives are counting in numbers, not in letters (e.g. /dev/da0, /dev/da1, etc).

Now, after the bugfix, the regular expression is directly awaited in the input:

./check_smart.pl -g "/dev/sd[a-z]" -i scsi

It actually makes much more sense this way because the end-user can define the regular expression (and therefore also create exceptions for the regex aware admins).

The new version 5.6, released as of now, includes this bugfix. Please adapt your service definitions accordingly (only required if you make use of -g of course).

 

Strange hex characters appear in html source code (check_http and chunks)
Tuesday - Mar 10th 2015 - by - (1 comments)

A website, being monitored with the Nagios plugin check_http, was constantly reporting an alert, because the string being looked for in the source code (footerContent) was not found:

./check_http -I 192.168.4.50 -H www.example.com -s footerContent
HTTP CRITICAL: HTTP/1.1 200 OK - string footerContent not found on http://www.example.com:80/ - 21322 bytes in 2.500 second response time

check_http offers a verbose option which displays the full source code. And by using the -v parameter, the source code revealed strange signs:

./check_http -I 192.168.4.50 -H www.example.com -v
[...]
                <div id="footer">
                        <div class="footerCo
305
ntent cfx">
                                <p>&copy; 2015 EXAMPLE</p>
                                 <ul><li class=""><a href="/en/site-links/terms-conditions/" target="_self" class="">Terms & conditions
[...]

Note the "305" string in the source code, which sits in the middle of the string "footerContent"? Now it makes sense, why the Nagios http check failed and couldn't find the full string.
At first we expected a bug in the check_http plugin but by using curl's --raw option, the same behavior could be seen:

curl http://192.168.4.50 -H "Host: www.example.com" -v --raw

So the question was: Where does this strange number come from? Another look at the full source code output of check_http revealed several other strange characters.

A tcpdump also revealed that the website was loaded in several tcp packets. One of the packets started with the content "ntent cfx". So right after the 305 number. It seemed that Nginx was splitting up the source code into several packets, the "305" was therefore the beginning of the new packet with its size (at least that's what we guessed). But why did this happen?

I enabled the nginx debug log and launched the check_http command again.
Meanwhile I had tried to add/remove several Nginx proxy parameters, so the weird character added in the source code has changed its position and value:

./check_http -I 192.168.4.50 -H www.example.com -v
[...]
                <div id="footer">
                        <div class="footerContent cfx">
                                <p>&copy; 2015 EXAMPLE</p>
                                 <ul
2d4
><li class=""><a href="/en/site-links/terms-conditions/" target="_self" class="">Terms & conditions
[...]

In the huge debug output I found the following information:

[...]
[debug] 43063#0: *531 HTTP/1.1 200 OK
Server: nginx/1.7.8
Date: Tue, 10 Mar 2015 12:04:21 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: close
X-Powered-By: PHP/5.5.9-1ubuntu4
Set-Cookie: CONCRETE5=hq9tb7d4kvqqn8bfdv9uvofgm1; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding
[...]

There was one very interesting line which caught my eye: Transfer-Encoding: chunked
Nginx therefore took the response from the upstream server and split it into several pieces (chunks).

I modified the Nginx vhost and disabled chunked_transfer_encoding:

    location / {
        proxy_pass http://upstream;
        chunked_transfer_encoding off;
    }

As soon as I reloaded Nginx, I tested it with check_http again:

./check_http -I 192.168.4.50 -H www.example.com -v
[...]
                <div id="footer">
                        <div class="footerContent cfx">
                                <p>&copy; 2015 EXAMPLE</p>
                                 <ul><li class=""><a href="/en/site-links/terms-conditions/" target="_self" class="">Terms & conditions

No strange characters, no breaking up the source code anymore! I went through the full source code and couldn't find any of these "code splits" anymore.
In the Nginx debug log, the output changed, too:

[debug] 43196#0: *655 HTTP/1.1 200 OK
Server: nginx/1.7.8
Date: Tue, 10 Mar 2015 12:11:46 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
X-Powered-By: PHP/5.5.9-1ubuntu4
Set-Cookie: CONCRETE5=it08ppn4bgjsjg1qvsvleo4432; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding

The Header "Transfer-Encoding" disappeared. So this is the reason, why these "strange characters" showed up in the html source code.

These characters don't show up in a browser, because they're correctly decrypted. However when "raw" http data is read then these chunks split up your source code, hence causing you an alert if your search string is cut by the chunk. So the issue here comes from the check_http plugin which uses HTTP/1.1 but does not understand the chunked http encoding. By definition, every client using HTTP/1.1 must understand Transfer-Encoding chunked.

 

Cannot read Audio CD in Hifi/Audio CD player but works on computer
Monday - Mar 9th 2015 - by - (0 comments)

I had to prepare an audio CD with a few songs on it. There were several CD-R (no brand, 700MB 80min 52x speed) empty disc's around so I just took one and burned the songs with CDBurnerXP.

Once the disc was finished, I tested it on the same computer and the CD played just fine. But when I tried it on a HiFi system the CD couldn't be read. I also tried it in my car - same there. The disc wasn't recognized.

I thought maybe that one empty disc was defect and tried it with one, two, three... others. Always the same issue. Then I read an information about the disc burning speed. I tried to burn another empty disc with 4x speed; same issue again.

Finally I tried it with a completely different set of empty discs (Sony CD-R 700MB 48x speed) and it immediately worked on the HiFi (I burned the disc with 4x speed).

Reminder to myself: Don't use no-brand empty discs for audio CD's.

 

When a Windows support phone scammer reaches a Linux Systems Engineer...
Tuesday - Mar 3rd 2015 - by - (0 comments)

The "Windows phone scams" are nothing new to the insiders of IT. I even know a person who got successfully scammed, obviously not an IT pro. But whether the scammers call IT aware people or just Internet users, the scamming is mean and criminal.

A few days ago, I got such a call from an American number. It all started with "I am calling you from Microsoft Windows support. Your computer has a virus." That sentence was enough that I knew this was the famous phone scam.

I quickly installed a voice recording app and recorded the phone conversation. The very beginning is missing but the most important part is there. The conversation can be found on Youtube on the following link:

Windows tech support phone scammer reaches a Linux Systems Engineer

https://www.youtube.com/watch?v=GIIpHt6cWzA

During the call I had to improvise (because I wasn't actually sitting at a computer) but I had two goals:

1) Stall the callers and let them waste time with me. The more time they waste on me, the less they can scam other people.
2) Find out how they react if the target (the user) is running a Linux computer.

Funnily I gave enough hints during the call, that my system is a Linux computer, but they were all ignored - until the very end. But listen for yourself.

 


Go to Homepage home
Linux Howtos how to's
Nagios Plugins nagios plugins
Links links

Valid HTML 4.01 Transitional
Valid CSS!
[Valid RSS]

8308 Days
until Death of Computers
Why?