Well what can I say about this? Not much really, but it would have been friggin funny if I saw a Bluescreen on it!

Strange behavior yesterday a newly installed ESXi 4.1 machine: The vmfs3 partition used only around 27% of the disk capabilities. Meaning: From the logical harddisk of 2.86 TB only 744GB were used for VMFS.

After some extended research and several tests, it looks like the ESXi 4.1 installer may have a bug in the partitioner, causing to add a primary partition for vmfs which is smaller than it could be. But with manual tweaks on the ESXi machine and (if necessary) in a partition tool like GParted, one can get rid of this bug by manually recreating the vmfs partitions. 

I wrote a How To called VMware ESXi 4.1 does not use whole disk capacities for VMFS3 which describes in details how to manually repartition and reformat the vmfs volumes on an ESXi 4.1 machine.

Update 29.07.2010:
Meanwhile I've contacted VMware to get some information from the 'source' why ESXi is doing this. The official response is that this is a 'wanted limitation' of ESXi and it is unlikely that this limitation will disappear in the near future. So if you want to use ESXi 4.1 with a lot of disk capacity you either may have to switch to ESX (for manual partitioning) or you follow my HowTo which serves as a workaround here.

But no. The Veeam jobs didn't run anymore and ended with an error:

Analyzing object "VM-Name" Object VM-Name not found. There are no objects to backup

What can be done about that is to go into the properties of each job and reselect the VM's to backup. This adds the new so called refID from the new inventory of VCenter Server into the Veeam database and should now communicate with the correct VM.

Important Update 20.07.2010:
Even though some manual adjustments were made in the job properties and even in the SQL database, the jobs won't run. After having contacted Veeam support, the answer is simple: Veeam Backup & Replication in version 4.1 is not compatible with vSphere vCenter Server 4.1!

Is it really that long ago? Have 4 years passed so fast? That is around the last time I had to install a new Windows 2003 Server from scratch on a Dell PowerEdge server. Back then it was a Power Edge 1850, yesterday it was a 1950. Pretty much the same history: Windows setup can't find the hard-disks and therefore aborts the installation.

4 years back I simply used a diskette with the LSI Logic drivers for SAS disk and of course I had this diskette ready on my desk for future use. Meanwhile I am at another company and all my hardware-preparations (CD's and diskette's) are gone. And now the bad news: The diskette link on the Dell site has disappeared! But here's what to do now, in 2010:

1. Download the Dell USB Key F6 Driver Utility, v.1.2.4, A08 (or newer) and install it (e.g. into C:\Program Files\Dell\USBKeyPrepF6).

2. Download the Dell Driver (not Firmware!!) for your SAS Raid controller from. This is in most cases the PERC 5/i or 6/i depending on your server model. The easiest method is to go to http://support.dell.com,  search your server model, select your wanted operating system (watch out between 32bit and 64bit!) and open the section SAS Raid Controller. Then click on the 'Dell - Driver' link and select the Hard Drive version of the driver. To help you find it quicker:
Windows 2003 32bit: DELL_MULTI-DEVICE_A07_R211422.exe
Windows 2003 64bit: DELL_MULTI-DEVICE_A07_R211424.exe

3. Execute the downloaded driver in point 2. A self-extract setup will launch and select the following folder to extract the files into: C:\Program Files\Dell\USBKeyPrepF6\Files (depending where you have installed the Dell Utility in point 1).

4. Now depending on your PERC version (which you can see on your boot screen of the server) you might need to change the file :\Program Files\Dell\USBKeyPrepF6\Files\txtsetup.oem. Follow the table below for which cases the line starting with 'scsi =' has to be changed:

Storage Controller        TXTSETUP.OEM needs to be changed?
Dell SAS 5/E Adapter            No change needed
Dell SAS 5/i Integrated            No change needed
Dell PERC 5/E Adapter            Change to:  scsi = PERC_32
Dell PERC 5/i Integrated        Change to:  scsi = PERC_32
Dell PERC 6/E Adapter            No change needed
Dell PERC 6/i Integrated        No change needed
LSI Logic CERC ATA 100            Change to:  scsi = PERC32
LSI Logic PERC 4/DC            Change to:  scsi = PERC32
LSI Logic PERC 4/Di            Change to:  scsi = PERC32
LSI Logic PERC 4/SC            Change to:  scsi = PERC32
LSI Logic PERC 4e/DC            Change to:  scsi = PERC32
LSI Logic PERC 4e/Di            Change to:  scsi = PERC32
LSI Logic PERC 4e/Si            Change to:  scsi = PERC32
LSI Logic PERC3/DC            Change to:  scsi = PERC32
LSI Logic PERC3/DCL            Change to:  scsi = PERC32
LSI Logic PERC3/QC            Change to:  scsi = PERC32
LSI Logic PERC3/SC            Change to:  scsi = PERC32
LSI Logic LSI2032            No change needed
Dell SAS 5x/6x Controller        No change needed
Adaptec U320 HostRAID Controller    No change needed
Adaptec CERC SATA 2s Controller        No change needed
Adaptec 39320A Controller        No change needed
Adaptec 39320A Leadfree Controller    No change needed

5. Now plug a USB memory stick up to 16GB into your computer and launch USBKeyPrepF6.exe. Warning: All the data on the memory stick will be deleted. The utility copies the files and formats the USB stick in a special way, which makes the stick able to be seen like a virtual floppy drive when booting on a Dell server. Don't worry, when you can't access the stick anymore by Windows, that's normal now.

6. Plug the memory stick in your Power Edge server and boot it. You have of course also inserted the Windows 2003 installation CD. So you boot from the CD and when Windows setup asks you at the very begin to hit the F6 key (for additional drivers) then hit it. After loading some drivers, setup is now asking if it should look for additional drivers. Hit the 'S'-key to search and the PERC drivers will be found on the plugged memory stick.

That's it! :-)

Today my Anti-Spam-Virus-Gateway running with Postfix/Amavis/ClamAV ran into a strange problem. It let through a virus in a exe file, which was packed as a zip file. When I installed this gateway, I tested this behavior of course and there were no problems at all. So there are two questions to be answered:

1) Why did the packed exe file go through (it should have been banned in the first place)?
2) Why wasn't the virus detected?

To answer the first question, I did some research in the logs which show that the zip file couldn't be uncompressed:

Jul  7 14:59:19 linux amavis[15983]: (15983-02) (!)Decoding of p003 (Zip archive data, at least v2.0 to extract) failed, leaving it unpacked: Compress::Raw::Zlib defines neither package nor VERSION--version check failed at (eval 79) line 467.

Now it makes sense, that the mail with the exe was not banned since Amavisd couldn't decompress the zip file to see that there is an exe in it! But why does that happen? It has worked before.

After some more and intensive version checkings and research I figured it must be due to a newer Amavisd-new version which I installed lately. And indeed, it is. I removed the current rpm amavisd-new-2.6.4-28.1 and installed the older version amavisd-new-2.6.3-8.1.x86_64.rpm:

rpm -e amavisd-new-2.6.4-28.1
rpm -ivh amavisd-new-2.6.3-8.1.x86_64.rpm

After that I resent from an external e-mail account a mail with a zip attachment containing an exe file and see what happens in the mail log:

Jul  7 16:51:58 linux amavis[32264]: (32264-01) p.path BANNED:1 recipient@example.com: "P=p004,L=1,M=multipart/mixed | P=p003,L=1/2,M=application/x-zip,T=zip,N=windirstat1_1_2_setup.zip | P=p006,L=1/2/1,T=exe,T=exe-ms,N=windirstat1_1_2_setup.exe", matching_key="(?i-xsm:.\\.(exe|msi|dll|vbs|pif|scr|bat|cmd|com|cpl)$)"

That's how it should be! So the problem comes definitely from Amavisd-new 2.6.4. Or at least from the rpm package for SLES10.

To answer the second question. This is a virus which was detected in the wild only today. This can unfortunately happen. For this case I'll install another virus scanner next to ClamAV with another signature base following the principle two eyes see better than one.

I would like to announce the immediate availability of check_esxi_wbem.py in version 20100705.

Aaron Rogers reprogrammed the Exit Msg/Exit Code logic on the HP server check. Before, the exit code was OK even though a critical message and exit code should have been the case (in his case a disk was defect). His correct logic has been added to version 20100702.
Since this logic was added, the plugin would only return UNKNOWN states for Dell servers which are OK. Therefore I added the same logic at the end of the Dell server check as well (20100705). This has been tested successfully on both HP and Dell servers. 

This weekend I updated a lot of ESXi servers to the latest version (build #261974) with the Host Update Utility 4.0. This went really smooth, no problems at all. Even bigger was the surprise, when I suddenly couldn't find the 'Hardware Status' tab in vSphere Client anymore.
A quick research on Google pointed me towards the Plugin-Manager (Plug-ins -> Plug-in Manager) and there it was: The vCenter Hardware Status plugin was disabled:

Looking at the error message, it seems that the vSphere Client couldn't correctly fetch the scriptConfig.xml file using port 8443 from the vCenter Server.
Another research pointed me to the information VMware KB 1010641: Enabling the Virtual Server Service status and hardware status plugins fails. This KB information confirmed it. One of the points in the checklist is to verify, that the computer on which vSphere Client is running can connect on port 8443 to the vCenter Server.

To solve it: Check your firewall settings and verify that port 8443 is open. If you can still not connect via a simple 'telnet vcserver 8443' don't blame the firewall yet. On the vCenter server check if the 'VMware VirtualCenter Management Webservices' service is running. It could be that this service is set to a manual startup type so you have to start it manually. Once this service is started, check the connection to port 8443 again. It should work now.

• Plugin outputs server information
  As we know it from other hardware plugins (like check_hpasm or check_openmanage) the check_esxi_wbem.py plugin now outputs server information as well (server model, serial number and bios information)
  
  
• Unknown set as default exit code
  The previous default exit code was 'OK'. So if something didn't work correctly, the plugin states 'OK' anyway. This has been fixed.
  
  
• Correct 'Authentication error' message
  Before, just a not very informative 'CRITICAL' error was shown when the authentication to the ESX/ESXi server was not correct. This patch now shows a correct 'Authentication error' message in the Nagios output.

550 5.7.1 RESOLVER.RST.AuthRequired; authentication required

By default, distribution groups are "made for internal purposes" and not for e-mails received by internet. Open the Exchange Management Console and open Recipient Configuration -> Distribution Group. Right-click on your distribution group and select Properties.
In the 'Mail Flow Settings' tab click on 'Message Delivery Restrictions' and click on Properties.
You will see, that 'Require that all senders are authenticated' is activated. Because emails from internet cannot be authenticated (by Active Directory obviously) they will not be accepted. As soon as you deactivate this option, e-mails from internet or other external sources will be accepted.