Header RSS Feed
 
If you only want to see the articles of a certain category, please click on the desired category below:
ALL Android Backup BSD Database Hacks Hardware Internet Linux Mail MySQL Nagios/Monitoring Network Personal PHP Proxy Shell Solaris Unix Virtualization VMware Windows Wyse

XML validation command line on Windows server
Monday - Sep 26th 2016 - by - (0 comments)

Was looking for a way to validate a xml file and came across a lot of posts of xmllint, which looked promising. However the Windows interpretation didn't seem to work correctly as it should have.

Eventually I came across XMLStarlet. It allows to be run on the command line and is able to validate and convert a XML file into other formats. Sounds good, but will it work?

First I tested the full xml file for its validation:

C:\Downloads>xml.exe --help
XMLStarlet Toolkit: Command line utilities for XML
Usage: xml.exe [<options>] <command> [<cmd-options>]
where <command> is one of:
  ed    (or edit)      - Edit/Update XML document(s)
  sel   (or select)    - Select data or query XML document(s) (XPATH, etc)
  tr    (or transform) - Transform XML document(s) using XSLT
  val   (or validate)  - Validate XML document(s) (well-formed/DTD/XSD/RelaxNG)
  fo    (or format)    - Format XML document(s)
  el    (or elements)  - Display element structure of XML document
  c14n  (or canonic)   - XML canonicalization
  ls    (or list)      - List directory as XML
  esc   (or escape)    - Escape special XML characters
  unesc (or unescape)  - Unescape special XML characters
  pyx   (or xmln)      - Convert XML into PYX format (based on ESIS - ISO 8879)
  p2x   (or depyx)     - Convert PYX into XML
<options> are:
  -q or --quiet        - no error output
  --doc-namespace      - extract namespace bindings from input doc (default)
  --no-doc-namespace   - don't extract namespace bindings from input doc
  --version            - show version
  --help               - show help
Wherever file name mentioned in command help it is assumed
that URL can be used instead as well.

Type: xml.exe <command> --help <ENTER> for command help

XMLStarlet is a command line toolkit to query/edit/check/transform
XML documents (for more information see http://xmlstar.sourceforge.net/)

C:\Downloads>xml.exe val xmlfile.xml
xmlfile.xml - valid

So far so good. But what happens if I change the xml file and remove some closing tag?

C:\Downloads>xml.exe val xmlfile.xml
xmlfile.xml - invalid

 Great, exactly what I needed!

 

Force Postfix to allow empty HELO/EHLO commands
Thursday - Sep 22nd 2016 - by - (0 comments)

As I previously wrote, I was working on a central SMTP relay system for internal servers. I came across a strange problem today, when a client couldn't send any mails.

After activating that particular client (10.10.44.20) in the debug_peer_list, I found the following information in the mail logs:

postfix/smtpd[9045]: connect from unknown[10.10.44.20]
postfix/smtpd[9045]: match_hostaddr: smtpd_client_event_limit_exceptions: 10.10.44.20 ~? cidr:/etc/postfix/networks(0,lock|utf8_request)
postfix/smtpd[9045]: dict_cidr_lookup: /etc/postfix/networks: 10.10.44.20
postfix/smtpd[9045]: > unknown[10.10.44.20]: 220 inf-smtp01-p.nzzmg.ch ESMTP Postfix (Ubuntu)
postfix/smtpd[9045]: < unknown[10.10.44.20]: HELO
postfix/smtpd[9045]: > unknown[10.10.44.20]: 501 Syntax: HELO hostname
postfix/smtpd[9045]: < unknown[10.10.44.20]: QUIT
postfix/smtpd[9045]: > unknown[10.10.44.20]: 221 2.0.0 Bye
postfix/smtpd[9045]: match_hostaddr: smtpd_client_event_limit_exceptions: 10.10.44.20 ~? cidr:/etc/postfix/networks(0,lock|utf8_request)
postfix/smtpd[9045]: dict_cidr_lookup: /etc/postfix/networks: 10.10.44.20

Interestingly, the client didn't send any value after the HELO command. That's a must according to the basic SMTP rules!

Without deeper explanation into the who and why's, I added a workaround into Postfix to allow such empty HELO commands. Or better said: I told Postfix to rewrite the faulty SMTP command.

This can be done by using smtpd_command_filter in /etc/postfix/main.cf:

smtpd_command_filter = pcre:/etc/postfix/command_filter

In order to use pcre tables, the pcre postfix module must first be installed:

apt-get install postfix-pcre

Within the /etc/postfix/command_filter file, I used the exact same example as mentioned in the official Postfix documentation for smtpd_command_filter:

# Work around clients that send malformed HELO commands.
/^HELO\s*$/ HELO domain.invalid

The empty HELO can now be tested manually with telnet:

$ telnet centralmailrelay.example.com 25
Trying xxx.xxx.xxx.xxx...
Connected to centralmailrelay.example.com.
Escape character is '^]'.
220 mail1.example.com ESMTP Postfix (Ubuntu)
HELO
250 mail1.example.com
quit
221 2.0.0 Bye
Connection closed by foreign host.

Yes. Works.

But, as mentioned, this should be fixed on the client as this is SMTP basics.

 

Ubuntu Xenial root partition filling up constantly due to kernel updates
Tuesday - Sep 20th 2016 - by - (1 comments)

So Ubuntu 16.04 (Xenial) was officially realeased in April 2016 and so far I'm running multiple servers, including production server, with this release.

One thing these machines have in common: The root partition seems to grow constantly. On the older trusty machines this didn't happen.

Xenial root partition filling up 

The reason for this? Xenial gets a lot of kernel updates and keeps every version, even minor version. This results in increasing disk space usage in /boot - which, in my case is usually in the root partition.

How to delete them? There's a new command "purge-old-kernels" which makes the cleanup for you:

root@xenial:~# purge-old-kernels
Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following packages were automatically installed and are no longer required:
  linux-headers-4.4.0-21 linux-headers-4.4.0-22 linux-headers-4.4.0-24 linux-headers-4.4.0-31 linux-headers-4.4.0-34
Use 'apt autoremove' to remove them.
The following packages will be REMOVED:
  linux-headers-4.4.0-21-generic* linux-headers-4.4.0-22-generic* linux-headers-4.4.0-24-generic* linux-headers-4.4.0-31-generic*
  linux-headers-4.4.0-34-generic* linux-image-4.4.0-21-generic* linux-image-4.4.0-22-generic* linux-image-4.4.0-24-generic*
  linux-image-4.4.0-31-generic* linux-image-4.4.0-34-generic* linux-image-extra-4.4.0-21-generic* linux-image-extra-4.4.0-22-generic*
  linux-image-extra-4.4.0-24-generic* linux-image-extra-4.4.0-31-generic* linux-image-extra-4.4.0-34-generic*
0 upgraded, 0 newly installed, 15 to remove and 61 not upgraded.
After this operation, 1,124 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 291145 files and directories currently installed.)
Removing linux-headers-4.4.0-21-generic (4.4.0-21.37) ...
Removing linux-headers-4.4.0-22-generic (4.4.0-22.40) ...
Removing linux-headers-4.4.0-24-generic (4.4.0-24.43) ...
Removing linux-headers-4.4.0-31-generic (4.4.0-31.50) ...
Removing linux-headers-4.4.0-34-generic (4.4.0-34.53) ...
Removing linux-image-extra-4.4.0-21-generic (4.4.0-21.37) ...
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.0-21-generic /boot/vmlinuz-4.4.0-21-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.0-21-generic /boot/vmlinuz-4.4.0-21-generic
update-initramfs: Generating /boot/initrd.img-4.4.0-21-generic
W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.
run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 4.4.0-21-generic /boot/vmlinuz-4.4.0-21-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 4.4.0-21-generic /boot/vmlinuz-4.4.0-21-generic
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.4.0-21-generic /boot/vmlinuz-4.4.0-21-generic
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.4.0-38-generic
Found initrd image: /boot/initrd.img-4.4.0-38-generic
Found linux image: /boot/vmlinuz-4.4.0-36-generic
Found initrd image: /boot/initrd.img-4.4.0-36-generic
Found linux image: /boot/vmlinuz-4.4.0-34-generic
Found initrd image: /boot/initrd.img-4.4.0-34-generic
Found linux image: /boot/vmlinuz-4.4.0-31-generic
Found initrd image: /boot/initrd.img-4.4.0-31-generic
Found linux image: /boot/vmlinuz-4.4.0-28-generic
Found initrd image: /boot/initrd.img-4.4.0-28-generic
Found linux image: /boot/vmlinuz-4.4.0-24-generic
Found initrd image: /boot/initrd.img-4.4.0-24-generic
Found linux image: /boot/vmlinuz-4.4.0-22-generic
Found initrd image: /boot/initrd.img-4.4.0-22-generic
Found linux image: /boot/vmlinuz-4.4.0-21-generic
Found initrd image: /boot/initrd.img-4.4.0-21-generic
done
Purging configuration files for linux-image-extra-4.4.0-21-generic (4.4.0-21.37) ...
Removing linux-image-4.4.0-21-generic (4.4.0-21.37) ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.4.0-21-generic /boot/vmlinuz-4.4.0-21-generic
update-initramfs: Deleting /boot/initrd.img-4.4.0-21-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 4.4.0-21-generic /boot/vmlinuz-4.4.0-21-generic
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.4.0-38-generic
Found initrd image: /boot/initrd.img-4.4.0-38-generic
Found linux image: /boot/vmlinuz-4.4.0-36-generic
Found initrd image: /boot/initrd.img-4.4.0-36-generic
Found linux image: /boot/vmlinuz-4.4.0-34-generic
Found initrd image: /boot/initrd.img-4.4.0-34-generic
Found linux image: /boot/vmlinuz-4.4.0-31-generic
Found initrd image: /boot/initrd.img-4.4.0-31-generic
Found linux image: /boot/vmlinuz-4.4.0-28-generic
Found initrd image: /boot/initrd.img-4.4.0-28-generic
Found linux image: /boot/vmlinuz-4.4.0-24-generic
Found initrd image: /boot/initrd.img-4.4.0-24-generic
Found linux image: /boot/vmlinuz-4.4.0-22-generic
Found initrd image: /boot/initrd.img-4.4.0-22-generic
done
Purging configuration files for linux-image-4.4.0-21-generic (4.4.0-21.37) ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.4.0-21-generic /boot/vmlinuz-4.4.0-21-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 4.4.0-21-generic /boot/vmlinuz-4.4.0-21-generic
Removing linux-image-extra-4.4.0-22-generic (4.4.0-22.40) ...
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.0-22-generic /boot/vmlinuz-4.4.0-22-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.0-22-generic /boot/vmlinuz-4.4.0-22-generic
update-initramfs: Generating /boot/initrd.img-4.4.0-22-generic
W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.
run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 4.4.0-22-generic /boot/vmlinuz-4.4.0-22-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 4.4.0-22-generic /boot/vmlinuz-4.4.0-22-generic
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.4.0-22-generic /boot/vmlinuz-4.4.0-22-generic
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.4.0-38-generic
Found initrd image: /boot/initrd.img-4.4.0-38-generic
Found linux image: /boot/vmlinuz-4.4.0-36-generic
Found initrd image: /boot/initrd.img-4.4.0-36-generic
Found linux image: /boot/vmlinuz-4.4.0-34-generic
Found initrd image: /boot/initrd.img-4.4.0-34-generic
Found linux image: /boot/vmlinuz-4.4.0-31-generic
Found initrd image: /boot/initrd.img-4.4.0-31-generic
Found linux image: /boot/vmlinuz-4.4.0-28-generic
Found initrd image: /boot/initrd.img-4.4.0-28-generic
Found linux image: /boot/vmlinuz-4.4.0-24-generic
Found initrd image: /boot/initrd.img-4.4.0-24-generic
Found linux image: /boot/vmlinuz-4.4.0-22-generic
Found initrd image: /boot/initrd.img-4.4.0-22-generic
done
Purging configuration files for linux-image-extra-4.4.0-22-generic (4.4.0-22.40) ...
Removing linux-image-4.4.0-22-generic (4.4.0-22.40) ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.4.0-22-generic /boot/vmlinuz-4.4.0-22-generic
update-initramfs: Deleting /boot/initrd.img-4.4.0-22-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 4.4.0-22-generic /boot/vmlinuz-4.4.0-22-generic
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.4.0-38-generic
Found initrd image: /boot/initrd.img-4.4.0-38-generic
Found linux image: /boot/vmlinuz-4.4.0-36-generic
Found initrd image: /boot/initrd.img-4.4.0-36-generic
Found linux image: /boot/vmlinuz-4.4.0-34-generic
Found initrd image: /boot/initrd.img-4.4.0-34-generic
Found linux image: /boot/vmlinuz-4.4.0-31-generic
Found initrd image: /boot/initrd.img-4.4.0-31-generic
Found linux image: /boot/vmlinuz-4.4.0-28-generic
Found initrd image: /boot/initrd.img-4.4.0-28-generic
Found linux image: /boot/vmlinuz-4.4.0-24-generic
Found initrd image: /boot/initrd.img-4.4.0-24-generic
done
Purging configuration files for linux-image-4.4.0-22-generic (4.4.0-22.40) ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.4.0-22-generic /boot/vmlinuz-4.4.0-22-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 4.4.0-22-generic /boot/vmlinuz-4.4.0-22-generic
Removing linux-image-extra-4.4.0-24-generic (4.4.0-24.43) ...
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.0-24-generic /boot/vmlinuz-4.4.0-24-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.0-24-generic /boot/vmlinuz-4.4.0-24-generic
update-initramfs: Generating /boot/initrd.img-4.4.0-24-generic
W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.
run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 4.4.0-24-generic /boot/vmlinuz-4.4.0-24-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 4.4.0-24-generic /boot/vmlinuz-4.4.0-24-generic
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.4.0-24-generic /boot/vmlinuz-4.4.0-24-generic
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.4.0-38-generic
Found initrd image: /boot/initrd.img-4.4.0-38-generic
Found linux image: /boot/vmlinuz-4.4.0-36-generic
Found initrd image: /boot/initrd.img-4.4.0-36-generic
Found linux image: /boot/vmlinuz-4.4.0-34-generic
Found initrd image: /boot/initrd.img-4.4.0-34-generic
Found linux image: /boot/vmlinuz-4.4.0-31-generic
Found initrd image: /boot/initrd.img-4.4.0-31-generic
Found linux image: /boot/vmlinuz-4.4.0-28-generic
Found initrd image: /boot/initrd.img-4.4.0-28-generic
Found linux image: /boot/vmlinuz-4.4.0-24-generic
Found initrd image: /boot/initrd.img-4.4.0-24-generic
done
Purging configuration files for linux-image-extra-4.4.0-24-generic (4.4.0-24.43) ...
Removing linux-image-4.4.0-24-generic (4.4.0-24.43) ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.4.0-24-generic /boot/vmlinuz-4.4.0-24-generic
update-initramfs: Deleting /boot/initrd.img-4.4.0-24-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 4.4.0-24-generic /boot/vmlinuz-4.4.0-24-generic
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.4.0-38-generic
Found initrd image: /boot/initrd.img-4.4.0-38-generic
Found linux image: /boot/vmlinuz-4.4.0-36-generic
Found initrd image: /boot/initrd.img-4.4.0-36-generic
Found linux image: /boot/vmlinuz-4.4.0-34-generic
Found initrd image: /boot/initrd.img-4.4.0-34-generic
Found linux image: /boot/vmlinuz-4.4.0-31-generic
Found initrd image: /boot/initrd.img-4.4.0-31-generic
Found linux image: /boot/vmlinuz-4.4.0-28-generic
Found initrd image: /boot/initrd.img-4.4.0-28-generic
done
Purging configuration files for linux-image-4.4.0-24-generic (4.4.0-24.43) ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.4.0-24-generic /boot/vmlinuz-4.4.0-24-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 4.4.0-24-generic /boot/vmlinuz-4.4.0-24-generic
Removing linux-image-extra-4.4.0-31-generic (4.4.0-31.50) ...
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.0-31-generic /boot/vmlinuz-4.4.0-31-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.0-31-generic /boot/vmlinuz-4.4.0-31-generic
update-initramfs: Generating /boot/initrd.img-4.4.0-31-generic
W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.
run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 4.4.0-31-generic /boot/vmlinuz-4.4.0-31-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 4.4.0-31-generic /boot/vmlinuz-4.4.0-31-generic
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.4.0-31-generic /boot/vmlinuz-4.4.0-31-generic
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.4.0-38-generic
Found initrd image: /boot/initrd.img-4.4.0-38-generic
Found linux image: /boot/vmlinuz-4.4.0-36-generic
Found initrd image: /boot/initrd.img-4.4.0-36-generic
Found linux image: /boot/vmlinuz-4.4.0-34-generic
Found initrd image: /boot/initrd.img-4.4.0-34-generic
Found linux image: /boot/vmlinuz-4.4.0-31-generic
Found initrd image: /boot/initrd.img-4.4.0-31-generic
Found linux image: /boot/vmlinuz-4.4.0-28-generic
Found initrd image: /boot/initrd.img-4.4.0-28-generic
done
Purging configuration files for linux-image-extra-4.4.0-31-generic (4.4.0-31.50) ...
Removing linux-image-4.4.0-31-generic (4.4.0-31.50) ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.4.0-31-generic /boot/vmlinuz-4.4.0-31-generic
update-initramfs: Deleting /boot/initrd.img-4.4.0-31-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 4.4.0-31-generic /boot/vmlinuz-4.4.0-31-generic
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.4.0-38-generic
Found initrd image: /boot/initrd.img-4.4.0-38-generic
Found linux image: /boot/vmlinuz-4.4.0-36-generic
Found initrd image: /boot/initrd.img-4.4.0-36-generic
Found linux image: /boot/vmlinuz-4.4.0-34-generic
Found initrd image: /boot/initrd.img-4.4.0-34-generic
Found linux image: /boot/vmlinuz-4.4.0-28-generic
Found initrd image: /boot/initrd.img-4.4.0-28-generic
done
Purging configuration files for linux-image-4.4.0-31-generic (4.4.0-31.50) ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.4.0-31-generic /boot/vmlinuz-4.4.0-31-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 4.4.0-31-generic /boot/vmlinuz-4.4.0-31-generic
Removing linux-image-extra-4.4.0-34-generic (4.4.0-34.53) ...
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.0-34-generic /boot/vmlinuz-4.4.0-34-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.0-34-generic /boot/vmlinuz-4.4.0-34-generic
update-initramfs: Generating /boot/initrd.img-4.4.0-34-generic
W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.
run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 4.4.0-34-generic /boot/vmlinuz-4.4.0-34-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 4.4.0-34-generic /boot/vmlinuz-4.4.0-34-generic
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.4.0-34-generic /boot/vmlinuz-4.4.0-34-generic
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.4.0-38-generic
Found initrd image: /boot/initrd.img-4.4.0-38-generic
Found linux image: /boot/vmlinuz-4.4.0-36-generic
Found initrd image: /boot/initrd.img-4.4.0-36-generic
Found linux image: /boot/vmlinuz-4.4.0-34-generic
Found initrd image: /boot/initrd.img-4.4.0-34-generic
Found linux image: /boot/vmlinuz-4.4.0-28-generic
Found initrd image: /boot/initrd.img-4.4.0-28-generic
done
Purging configuration files for linux-image-extra-4.4.0-34-generic (4.4.0-34.53) ...
Removing linux-image-4.4.0-34-generic (4.4.0-34.53) ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.4.0-34-generic /boot/vmlinuz-4.4.0-34-generic
update-initramfs: Deleting /boot/initrd.img-4.4.0-34-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 4.4.0-34-generic /boot/vmlinuz-4.4.0-34-generic
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.4.0-38-generic
Found initrd image: /boot/initrd.img-4.4.0-38-generic
Found linux image: /boot/vmlinuz-4.4.0-36-generic
Found initrd image: /boot/initrd.img-4.4.0-36-generic
Found linux image: /boot/vmlinuz-4.4.0-28-generic
Found initrd image: /boot/initrd.img-4.4.0-28-generic
done
Purging configuration files for linux-image-4.4.0-34-generic (4.4.0-34.53) ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.4.0-34-generic /boot/vmlinuz-4.4.0-34-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 4.4.0-34-generic /boot/vmlinuz-4.4.0-34-generic

Yes, these were quite a few versions which were deleted.

 

Headscratcher: Server should be SSL-aware but has no certificate configured
Wednesday - Sep 14th 2016 - by - (0 comments)

Today I came across a very annoying problem which seemed so simple yet I was looking way too far to see the solution right in front of me.

When I set up a new virtual host on an Apache 2.4 webserver (running on Ubuntu 14.04). Besides the normal http vhost, I also added a second virtual host for SSL with the following ssl configuration:

<VirtualHost *:443>
  ServerName my.example.com
  DocumentRoot /srv/www/example

  <Directory /srv/www/example>
    Require all granted
    Options +FollowSymLinks
    AllowOverride All
  </Directory>

  # Logging
  ErrorLog /var/log/apache2/example.error.log
  CustomLog /var/log/apache2/example.access.log combined

  <IfModule mod_ssl.c>
  # SSL
  SSLEngine on
  SSLCertificateFile "/etc/apache2/ssl.crt/mycert.crt"
  SSLCertificateKeyFile "/etc/apache2/ssl.key/mycert.key"
  SSLCACertificateFile "/etc/apache2/ssl.crt/mycert.ca.crt"
  </IfModule>

</VirtualHost>

I then realized that this was the first SSL config on this Apache webserver, so I still had to enable the ssl module. And this is where the problems started.

# a2enmod ssl
Considering dependency setenvif for ssl:
Module setenvif already enabled
Considering dependency mime for ssl:
Module mime already enabled
Considering dependency socache_shmcb for ssl:
Module socache_shmcb already enabled
Enabling module ssl.
See /usr/share/doc/apache2/README.Debian.gz on how to configure SSL and create self-signed certificates.
To activate the new configuration, you need to run:
  service apache2 restart

# service apache2 restart
 * Restarting web server apache2
Action 'start' failed.
The Apache error log may have more information.

I checked the error log and found the following information:

[Wed Sep 14 08:29:27.585173 2016] [ssl:emerg] [pid 28482] AH02240: Server should be SSL-aware but has no certificate configured [Hint: SSLCertificateFile] ((null):0)
[Wed Sep 14 08:29:27.585183 2016] [ssl:emerg] [pid 28482] AH02312: Fatal error initialising mod_ssl, exiting.

I immediately disabled the ssl module again and started to scratch my head. I did A LOT of Apache SSL configurations over the past 10 years, yet I've never seen this one before. I googled and there are a lot of different solutions - yet all point out that there must be a problem in the SSL configuration. I triple-checked my ssl config, even copy-pasted a working SSL config from another Apache webserver. No success.
I then modified ports.conf to make sure the listener port 443 is also enabled for mod_ssl.c. Turned out it was already enabled for ssl_module, which got me confused because, If I remember correctly, this was called mod_ssl in Apache 2.2.
So no results from ports.conf, the default seems to be correct already. Talking of default, I checked out the default vhosts and indeed, the default-ssl.conf was not enabled. I checked that file out but didn't see anything which would explain why THIS would make it work compared to my vhost config but I enabled it anyway:

/etc/apache2/sites-enabled # ln -s ../sites-available/default-ssl.conf 001-default-ssl.conf

# apache2ctl configtest
Syntax OK

# service apache2 restart
 * Restarting web server apache2

I enabled the ssl module again, restarted Apache and here we go again:

# service apache2 restart
 * Restarting web server apache2
Action 'start' failed.
The Apache error log may have more information.

WTF?! This is the point where I asked myself what exactly am I missing. I must be missing something obvious, yet I wasn't able to see it. Even when I set Apache's logging to trace2, there was not really any helpful hint in the error logs:

[Wed Sep 14 08:37:29.170593 2016] [ssl:trace2] [pid 29664] ssl_engine_rand.c(124): Init: Seeding PRNG with 656 bytes of entropy
[Wed Sep 14 08:37:29.170663 2016] [ssl:info] [pid 29664] AH02200: Loading certificate & private key of SSL-aware server 'webserver.example.com:443'
[Wed Sep 14 08:37:29.170667 2016] [ssl:emerg] [pid 29664] AH02240: Server should be SSL-aware but has no certificate configured [Hint: SSLCertificateFile] ((null):0)
[Wed Sep 14 08:37:29.170671 2016] [ssl:emerg] [pid 29664] AH02312: Fatal error initialising mod_ssl, exiting.

Eventually I came across yet another serverfault question which basically mentions the same ssl problem. Although I couldn't find a solution there, I picked up a hint, to verify all existing (other) virtual hosts with apache2ctl:

# apache2ctl -t -D DUMP_VHOSTS
VirtualHost configuration:
*:443                  somethingelse.example.com (/etc/apache2/sites-enabled/somethingelse.example.com.conf:1)
*:80                   is a NameVirtualHost
         default server webserver.example.com (/etc/apache2/sites-enabled/000-default.conf:1)
         port 80 namevhost webserver.example.com (/etc/apache2/sites-enabled/000-default.conf:1)
         port 80 namevhost www.example.com (/etc/apache2/sites-enabled/example.com.conf:1)
                 alias test.example.com
[...]

And there we had it. In the very first line a SSL-listener 443 for somethingelse.example.com was defined.
I checked the this virtual host configuration and it looked like this:

# cat /etc/apache2/sites-enabled/somethingelse.example.com.conf
<VirtualHost *:443>
  ServerName  somethingelse.example.com

  <ifmodule mod_rewrite.c="">
  RewriteEngine On
  RewriteCond %{HTTPS} on
  RewriteRule (.*) http://%{HTTP_HOST}%{REQUEST_URI}
  </ifmodule>

</VirtualHost>

[...]

I immediately saw that this is not at all a correct ssl configuration. I removed that whole section after confirmation from my colleague that he was working on it but it's not needed anymore (because it didn't work, duh) and tried again:

# a2enmod ssl
Considering dependency setenvif for ssl:
Module setenvif already enabled
Considering dependency mime for ssl:
Module mime already enabled
Considering dependency socache_shmcb for ssl:
Module socache_shmcb already enabled
Enabling module ssl.
See /usr/share/doc/apache2/README.Debian.gz on how to configure SSL and create self-signed certificates.
To activate the new configuration, you need to run:
  service apache2 restart

# service apache2 restart
 * Restarting web server apache2    

Yes! Now it worked.

Conclusion: The error message "AH02240: Server should be SSL-aware but has no certificate configured [Hint: SSLCertificateFile] ((null):0)" indeed tells you there is something wrong in your SSL configuration but unfortunately doesn't point you to the very file which is causing the error. Dig (or better grep in this case) through all config files to find the bad one.

 

check_es_store is now check_es_system and also checks memory usage
Wednesday - Sep 7th 2016 - by - (0 comments)

When I released the monitoring plugin check_es_store back in June this year, I didn't really think I'd need something else than the disk check.

Yet a couple of days ago we've hit a production downtime which was traced back to our ElasticSearch cluster running in the cloud. The reason: ES ran out of memory and started to run garbage collection (gc) which slowed down ES a great deal. 

Lesson learned. I added a memory usage check in the plugin. Because the plugin now does more than to check the storage (hence store in the name), I renamed the plugin to check_es_system (to check the underlying system). 

In order to launch the disk or mem check, a new parameter (-t for checktype) was added.

Please check out the documentation of the monitoring plugin check_es_system for more information.

 

Remove old and install new version of Dell OMSA on CentOS
Tuesday - Sep 6th 2016 - by - (0 comments)

On a Dell PE R720, running a CentOS 6, I tried to install the newest Dell OMSA (OpenManage Server Administrator) 8.3 however the "normal" way with the tar.gz package and setup.sh didn't work, because the OS of CentOS was not "detected".

I already thought I'd have to tamper with the shellscripts to tell the setup it should use the rpm's for RHEL, I came across a very good article which uses a yum repository from Dell to install OMSA. That's handy!

However when I tried to install srvadmin-all, I got a lot of dependency resolving errors. I first needed to manually remove all the existing OMSA packages.
First I got the list of all dell* packages:

# rpm -qa |grep dell | tr '\n' ' '
dell_ie_ps-3.1.0-4.123.1.el6.x86_64 dell_ft_ie_interface-1.0.14-4.15.4.el6.noarch dell_ie_nitrogen-2.0.0-4.1.2.el6.x86_64 dell_ie_idrac7-2.0.0-4.1.4.el6.x86_64 dell-omsa-repository-2-5.noarch yum-dellsysid-2.2.27-4.12.1.el6.x86_64 dell_ie_sas-3.2.0-4.196.2.el6.x86_64 dell_ie_bios-3.1.0-4.125.1.el6.x86_64 dell_ie_maser_inv_lcl-3.2.0-4.130.1.el6.x86_64 dell_ie_tape_quantum-1.1.0-7.x86_64 dell_ie_tape_ibm-1.1.0-7.x86_64 dell_ie_nvme_pcissd-1.0.0-4.91.2.el6.x86_64 dell_ft_install-1.1-2.noarch firmware-addon-dell-2.2.9-1.el6.x86_64 dell_ie_maser-3.2.0-4.130.1.el6.x86_64 dell_ie_rac_5-7.4.0-4.1.158.el6.x86_64 dell_ie_pcissd-1.0.0-4.130.3.el6.x86_64

And then the same for the srvadmin* packages:

# rpm -qa |grep srvadmin | tr '\n' ' '
srvadmin-racadm4-7.4.0-4.1.158.el6.x86_64 srvadmin-idrac-ivmcli-7.4.0-4.5.3.el6.x86_64 srvadmin-xmlsup-7.4.0-4.24.1.el6.x86_64 srvadmin-storelib-7.4.0-4.177.1.el6.x86_64 srvadmin-omacs-7.4.0-4.97.1.el6.x86_64 srvadmin-racdrsc-7.4.0-4.12.6.el6.x86_64 srvadmin-smweb-7.4.0-4.152.2.el6.x86_64 srvadmin-server-cli-7.4.0-4.1.1.el6.x86_64 srvadmin-rac5-7.4.0-4.1.157.el6.x86_64 srvadmin-rac4-populator-7.4.0-4.1.158.el6.x86_64 srvadmin-idrac-7.4.0-4.12.6.el6.x86_64 srvadmin-rnasoap-7.4.0-4.152.1.el6.x86_64 srvadmin-nvme-7.4.0-4.178.1.el6.x86_64 srvadmin-storage-cli-7.4.0-4.152.2.el6.x86_64 srvadmin-hapi-7.4.0-4.28.2.el6.x86_64 srvadmin-sysfsutils-7.4.0-4.1.1.el6.x86_64 srvadmin-storelib-sysfs-7.4.0-4.1.1.el6.x86_64 srvadmin-deng-7.4.0-4.14.1.el6.x86_64 srvadmin-rac-components-7.4.0-4.12.6.el6.x86_64 srvadmin-ominst-7.4.0-4.97.1.el6.x86_64 srvadmin-omacore-7.4.0-4.97.1.el6.x86_64 srvadmin-omcommon-7.4.0-4.97.1.el6.x86_64 srvadmin-cm-7.4.0-4.1.115.el6.x86_64 srvadmin-server-snmp-7.4.0-4.1.1.el6.x86_64 srvadmin-isvc-snmp-7.4.0-4.42.2.el6.x86_64 srvadmin-racadm5-7.4.0-4.1.158.el6.x86_64 srvadmin-jre-7.4.0-4.98.1.el6.x86_64 srvadmin-webserver-7.4.0-4.1.1.el6.x86_64 srvadmin-oslog-7.4.0-4.100.1.el6.x86_64 srvadmin-idrac-vmcli-7.4.0-4.10.1.el6.x86_64 srvadmin-rac4-7.4.0-4.1.158.el6.x86_64 srvadmin-itunnelprovider-7.4.0-4.14.1.el6.x86_64 srvadmin-realssd-7.4.0-4.178.1.el6.x86_64 srvadmin-storage-7.4.0-4.152.2.el6.x86_64 srvadmin-storageservices-snmp-7.4.0-4.1.1.el6.x86_64 srvadmin-storageservices-cli-7.4.0-4.1.1.el6.x86_64 srvadmin-all-7.4.0-4.1.1.el6.x86_64 srvadmin-smcommon-7.4.0-4.152.2.el6.x86_64 srvadmin-argtable2-7.4.0-4.2.1.el6.x86_64 srvadmin-omilcore-7.4.0-4.100.1.el6.x86_64 srvadmin-isvc-7.4.0-4.42.2.el6.x86_64 srvadmin-deng-snmp-7.4.0-4.14.1.el6.x86_64 srvadmin-base-7.4.0-4.2.1.el6.x86_64 srvadmin-idrac-snmp-7.4.0-4.12.6.el6.x86_64 srvadmin-tomcat-7.4.0-4.97.1.el6.x86_64 srvadmin-idracadm-7.4.0-4.12.6.el6.x86_64 srvadmin-racsvc-7.4.0-4.1.158.el6.x86_64 srvadmin-standardAgent-7.4.0-4.1.1.el6.x86_64 srvadmin-storage-snmp-7.4.0-4.152.2.el6.x86_64 srvadmin-storageservices-7.4.0-4.1.1.el6.x86_64

Additionally to both lists, you also need to include the libsmal0 package, in my case libsmal0-3.1.0-4.146.1.el6.x86_64, because this is also a dependency of srvadmin-hapi.

So the complete uninstall command turned out to be:

# rpm -e srvadmin-racadm4-7.4.0-4.1.158.el6.x86_64 srvadmin-idrac-ivmcli-7.4.0-4.5.3.el6.x86_64 srvadmin-xmlsup-7.4.0-4.24.1.el6.x86_64 srvadmin-storelib-7.4.0-4.177.1.el6.x86_64 srvadmin-omacs-7.4.0-4.97.1.el6.x86_64 srvadmin-racdrsc-7.4.0-4.12.6.el6.x86_64 srvadmin-smweb-7.4.0-4.152.2.el6.x86_64 srvadmin-server-cli-7.4.0-4.1.1.el6.x86_64 srvadmin-rac5-7.4.0-4.1.157.el6.x86_64 srvadmin-rac4-populator-7.4.0-4.1.158.el6.x86_64 srvadmin-idrac-7.4.0-4.12.6.el6.x86_64 srvadmin-rnasoap-7.4.0-4.152.1.el6.x86_64 srvadmin-nvme-7.4.0-4.178.1.el6.x86_64 srvadmin-storage-cli-7.4.0-4.152.2.el6.x86_64 srvadmin-hapi-7.4.0-4.28.2.el6.x86_64 srvadmin-sysfsutils-7.4.0-4.1.1.el6.x86_64 srvadmin-storelib-sysfs-7.4.0-4.1.1.el6.x86_64 srvadmin-deng-7.4.0-4.14.1.el6.x86_64 srvadmin-rac-components-7.4.0-4.12.6.el6.x86_64 srvadmin-ominst-7.4.0-4.97.1.el6.x86_64 srvadmin-omacore-7.4.0-4.97.1.el6.x86_64 srvadmin-omcommon-7.4.0-4.97.1.el6.x86_64 srvadmin-cm-7.4.0-4.1.115.el6.x86_64 srvadmin-server-snmp-7.4.0-4.1.1.el6.x86_64 srvadmin-isvc-snmp-7.4.0-4.42.2.el6.x86_64 srvadmin-racadm5-7.4.0-4.1.158.el6.x86_64 srvadmin-jre-7.4.0-4.98.1.el6.x86_64 srvadmin-webserver-7.4.0-4.1.1.el6.x86_64 srvadmin-oslog-7.4.0-4.100.1.el6.x86_64 srvadmin-idrac-vmcli-7.4.0-4.10.1.el6.x86_64 srvadmin-rac4-7.4.0-4.1.158.el6.x86_64 srvadmin-itunnelprovider-7.4.0-4.14.1.el6.x86_64 srvadmin-realssd-7.4.0-4.178.1.el6.x86_64 srvadmin-storage-7.4.0-4.152.2.el6.x86_64 srvadmin-storageservices-snmp-7.4.0-4.1.1.el6.x86_64 srvadmin-storageservices-cli-7.4.0-4.1.1.el6.x86_64 srvadmin-all-7.4.0-4.1.1.el6.x86_64 srvadmin-smcommon-7.4.0-4.152.2.el6.x86_64 srvadmin-argtable2-7.4.0-4.2.1.el6.x86_64 srvadmin-omilcore-7.4.0-4.100.1.el6.x86_64 srvadmin-isvc-7.4.0-4.42.2.el6.x86_64 srvadmin-deng-snmp-7.4.0-4.14.1.el6.x86_64 srvadmin-base-7.4.0-4.2.1.el6.x86_64 srvadmin-idrac-snmp-7.4.0-4.12.6.el6.x86_64 srvadmin-tomcat-7.4.0-4.97.1.el6.x86_64 srvadmin-idracadm-7.4.0-4.12.6.el6.x86_64 srvadmin-racsvc-7.4.0-4.1.158.el6.x86_64 srvadmin-standardAgent-7.4.0-4.1.1.el6.x86_64 srvadmin-storage-snmp-7.4.0-4.152.2.el6.x86_64 srvadmin-storageservices-7.4.0-4.1.1.el6.x86_64 dell_ie_ps-3.1.0-4.123.1.el6.x86_64 dell_ft_ie_interface-1.0.14-4.15.4.el6.noarch dell_ie_nitrogen-2.0.0-4.1.2.el6.x86_64 dell_ie_idrac7-2.0.0-4.1.4.el6.x86_64 dell-omsa-repository-2-5.noarch yum-dellsysid-2.2.27-4.12.1.el6.x86_64 dell_ie_sas-3.2.0-4.196.2.el6.x86_64 dell_ie_bios-3.1.0-4.125.1.el6.x86_64 dell_ie_maser_inv_lcl-3.2.0-4.130.1.el6.x86_64 dell_ie_tape_quantum-1.1.0-7.x86_64 dell_ie_tape_ibm-1.1.0-7.x86_64 dell_ie_nvme_pcissd-1.0.0-4.91.2.el6.x86_64 dell_ft_install-1.1-2.noarch firmware-addon-dell-2.2.9-1.el6.x86_64 dell_ie_maser-3.2.0-4.130.1.el6.x86_64 dell_ie_rac_5-7.4.0-4.1.158.el6.x86_64 dell_ie_pcissd-1.0.0-4.130.3.el6.x86_64 libsmal0-3.1.0-4.146.1.el6.x86_64

Yep. That's a lot of packages. Yet it worked without any problems (some warnings about some ini files were shown, but they can be ignored). As you can see in the srvadmin* version numbers, the previous installed OMSA version was 7.4.

Now to the installation of the newer OMSA 8.3 version. First let's add the Dell repository:

wget -q -O - http://linux.dell.com/repo/hardware/latest/bootstrap.cgi | bash
Downloading GPG key: http://linux.dell.com/repo/hardware/latest/public.key
    Key already exists in RPM, skipping
Write repository configuration
Done!

This actually just adds this file on the system and enables the GPG key for using yum:

 # cat /etc/yum.repos.d/dell-system-update.repo
[dell-system-update_independent]
name=dell-system-update_independent
baseurl=http://linux.dell.com/repo/hardware/latest/os_independent/
gpgcheck=1
gpgkey=http://linux.dell.com/repo/hardware/latest/public.key
enabled=1    
exclude=dell-system-update*.i386

[dell-system-update_dependent]
name=dell-system-update_dependent
mirrorlist=http://linux.dell.com/repo/hardware/latest/mirrors.cgi?osname=el$releasever&basearch=$basearch&native=1
gpgcheck=1
gpgkey=http://linux.dell.com/repo/hardware/latest/public.key
enabled=1    

Now that we got the Dell yum repo enabled, we can conveniently install srvadmin from it:

yum -y install srvadmin-all

At the end I restarted the services:

# /etc/init.d/dataeng restart
Stopping Systems Management Data Engine:
Stopping dsm_sa_snmpd:                                     [  OK  ]
Stopping dsm_sa_eventmgrd: Not started                     [FAIL]
Stopping dsm_sa_datamgrd: Not started                      [FAIL]
Starting Systems Management Device Drivers:
Starting dell_rbu:                                         [  OK  ]
Starting ipmi driver:
                                                           [  OK  ]
Starting Systems Management Data Engine:
Starting dsm_sa_datamgrd:                                  [  OK  ]
Starting dsm_sa_eventmgrd:                                 [  OK  ]
Starting dsm_sa_snmpd:                                     [  OK  ]

The stopping failed because I uninstalled the old packages (I forgot to stop the OMSA services). The start of the services worked, OMSA 8.3 up and running:

# omreport about

Product name : Dell OpenManage Server Administrator
Version      : 8.3.0
Copyright    : Copyright (C) Dell Inc. 1995-2015 All rights reserved.
Company      : Dell Inc.

# omreport chassis
Health

Main System Chassis

SEVERITY : COMPONENT
Ok       : Fans
Ok       : Intrusion
Ok       : Memory
Ok       : Power Supplies
Ok       : Power Management
Ok       : Processors
Ok       : Temperatures
Ok       : Voltages
Ok       : Hardware Log
Ok       : Batteries

For further help, type the command followed by -?

 

AMD share in 2016: Doubting the so called financial technical analysts
Friday - Sep 2nd 2016 - by - (0 comments)
OK, let me start off with: I'm not in the finance sector. Yet I've seen so many wrong and misleading articles from so-called "technical specialists", "fi-tech advisors" or "tech analysts" in the recent months, I just have to rant about it. 

Earlier this year, AMD has hit a low of below 2 bucks a share. Back in January, the share traded as low as $1.80.

AMD January 2016

Clearly one of the lowest points (not even a 52w-low, but still) in the stock history. Nevertheless anyone in technology for more than 10 years knows that AMD is, besides Intel, the only x86 chip maker. What would happen, if AMD would go down? Intel would be the only x86 maker. That means: Market domination/monopoly. In 1997 there was a similar case. Remember when Apple almost got broke? Well it received help from *drumrolls* Microsoft. Microsoft appeared as the savior of Apple but there was actually a lot at stake for Microsoft if Apple went out of business. To quote the article linked before:

But Microsoft, for its $150 million, gets something in return:

-- A shield from the heat of antitrust regulators. If Apple ceases operation, Microsoft has a 100 percent share of the personal-computer market.

"From Microsoft's standpoint, I think their interests are pretty much motivated by antitrust concerns," said Richard Scocozza, a software analyst at Brown Brothers Harriman. "It's in their best interest to keep Apple from failing so they don't incur any additional scrutiny from the Justice Department."

That's right. Everybody wants to be the market leader - but if there's no market anymore, big troubles (= regulations) arise.
See the similarities to Intel/AMD? So AMD cannot go down, not without having received financial help first. And this didn't happen either. AMD sold their manufacturing branch and got some cash in, correct, but active financial aid (let's say from a government) was never received. AMD still had a pile of money from the good old 90's laying around to cover that. And even if there wasn't any money anymore, Intel would probably do the same as Microsoft did twenty years ago.

So far to the CPU market. In the graphics/GPU market, AMD clearly had a crappy few years and lost huge market shares to Nvidia. AMD, when it took over ATI, was probably the biggest and best known graphic card producer back then, but Nvidia came in with hyperspeed and eventually overlapped AMD. 2014-2015 it got even worse, when already leading Nvidia picked up speed and AMD lost once again:

AMD vs Nvidia 

Image from: http://marketrealist.com/2016/01/nvidia-snags-amds-share-gaming-market/

But AMD has learned from its mistakes or at least recognized that it cannot beat Intel in CPU chips anymore. So already in 2012, AMD decided to license ARM - the "alternative" CPU maker for mobile and embedded devices. And this was actually a crucial decision. With this license, AMD was able to create chips with the mobile and power-effcient ARM technology. With the in depth knowledge of making x86 chips (CPU) and graphic cards (GPU), who'd be in a better position than AMD to create computing chips, cpu and gpu bundled together, with the new ARM technology? No one. And this is where AMD's healing point would start.

And then, something else happened: Virtual Reality or VR. I'm not a game fanatic, never was, but this is something which is going to change the world. Did you play ego-shooters (like Unreal Tournament)? Did you ever play Paintball? Now put those two together and you will see the possibilities of VR. I have no doubt, that in the near future (up to 10 years) we're going into "game halls", wearing VR goggles, running around with laser tag guns and seeing our allies and enemies in the game through the goggles. It doesn't matter if your team mate is playing along side with you, I mean standing in reality next to you, or not. Increasing Internet speeds will allow gaming remotely. Probably in game halls with the same shapes to reflect the maps of the game, but who knows with what they will come up. Anyway, the point is: Virtual Reality will come and it'll be big. And what exactly is producing the in-game-experience? The goggles. This means: A very fast CPU and GPU running in the googles. And once again, who is able to put these two together in the best way? AMD. And this is where AMD's big-announced "Zen" technology comes in play. Massive CPU speed (comparable to Intel's rocket launcher Xenon), bundled together with a powerful GPU. That's not only for the classical gaming market something new, it will be a requirement for the new kind of VR-gaming.

So there are several reasons, why AMD will definitely not die. And even if they'd get out of cash, some intelligent investors would come to help because the potential of the emerging VR market is huge. Really huge (see Consumer Virtual Reality market worth $5.2 bn by 2018).

Yet, whenever I checked on the current share price on Google Finance, I saw disturbing news that AMD will go down, that AMD's shares are worthless etc. Let's pin that down:

  -------------------------------------------

1) Ashraf Eassa from The Motley Fool wrote in January 2014, that you better "Don't Buy AMD's ARM Server Story". Why? Reasons were given that

"ARM is disrupting x86 servers"

Which is true but aims at another target market

"ARM servers will offer choice"

So it's 2016 and what kind of ARM servers are available? Very few. What choice are we talking about here?
And another one: 

"ARM is lower power"

Yes, of course, but consumes less energy. That's the whole point! It's perfect for small/embedded devices.

In January 2014 AMD first rose to over $4 but then fell to ~$3.40 to then rise to almost $3.90 again.

2) Leo Sun asked on The Motley Fool in January 2016 "Can Advanced Micro Devices Inc. Survive 2016?". Given the share price back then of under $2, it looked indeed pretty dark for AMD. The article correctly states that market share fell against AMD's competitors. But once again, AMD is only compared against Intel and Nvidia. The ARM stategy of embedded devices is only stated with:

Instead of competing more aggressively against Intel and Nvidia, AMD has been counting on its EESC (Enterprise, Embedded, and Semi-Custom) unit to offset those losses. That division produces SoCs for gaming consoles, servers, and other embedded devices. Unfortunately, revenue at that unit fell 15% annually to $488 million last quarter, while its operating income plummeted 46% to $59 million. AMD's guidance indicates that growth likely won't bounce back unless console sales accelerate -- which could be tough considering that the PS4 and Xbox One are already over two years old.

Instead of seeing the embedded market as AMD's chance of rising again, the analyst suggested to attack Intel once more:

Looking ahead, AMD's best hope is to play offense against Intel with its new Zen x86 processors

No word of the emerging VR market, however a correct response to the question in the title of the article:

However, AMD likely won't go bankrupt in 2016. The company's cash and equivalents dipped just 2.5% annually to $785 million last quarter, and its total debt only rose 2.3% to $2.26 billion. No new debt should come due until 2019. [...] Therefore, the real question investors should ask isn't if AMD can survive 2016, but whether it will still be around in 2020.

3) In April 2016, Chris Neiger wrote on The Motley Fool the article "NVIDIA vs. AMD: Which Is the Better Buy?" and came to the conclusion:

"Both companies clearly have their strong suits, but I think NVIDIA is a better buy right now because of the new market segments it's pursuing. There are projected to be 12 million driverless car sales annually by 2035, according to IHS Automotive, and NVIDIA's technology is helping to push that market forward in ways AMD just can't match. Similarly, VR's growth over the coming years should bring in additional revenue for both companies, but NVIDIA's dominant position in GPU market share gives it the clear advantage in the space. AMD may be expanding further into servers and PC processors, but just as Intel has shown, betting on the PC market is a losing battle. I think NVIDIA's pursuits have better long-term potential, and that's why I'd choose it right now."

So first of all: Nvidia has much more to lose in the GPU market. As it already has a high market share, it's more likely this share will go down than up.
Second of all: The article talks about potential driverless cars - in 2035! We will see VR before that.
Third point: He's absolutely right that VR's growth will be a booster. But here again comes AMD with its CPU+GPU bundle experience plus the licensed ARM technology. Pole position at least.
And lastly: The author is putting into the "classic" role to compete against Intel again. Sorry mate, AMD already knew when they obtained the ARM license, that the pure CPU game against Intel is over.

Note that in April 2016, AMD shares umped from ~$2.60 to ~$3.60:

AMD April 2016

4) Timothy Green from The Motley Fool wrote on Jun 22nd 2016 "3 Reasons Advanced Micro Devices Inc. Stock Could Fall". This is probably THE article which stuck in my head and which led me to write this rant. There are so many personal opinions in this article, none with technical proof or sight as:

"This kind of optimism can be dangerous, especially given AMD's history of failing to deliver on its promises."

NVIDIA's unit share of the graphics card market stands at about 80%, up from 60% in early 2014. [...] If AMD doesn't produce the kind of market share gains investors seem to be expecting, the stock could lose some of its recent luster.

"With a current market capitalization of around $4 billion, buying AMD at this price requires the belief that AMD is not only going to surge back to profitability, but that the company is going to be fundamentally different than it has been in the past. The stock price may have gotten out ahead of what AMD can realistically accomplish, and any bad news at all could send shares lower."

The article itself has a lot of "could" in it, but the title itself got on my nerves. If you only read the title, you assume that AMD will fall and you have to short the stock. When the author mentioned "buying AMD at this price..." (which was at that day on June 22nd 2016 at $5) "... requires the belief that AMD is not only going to surge back to profitability...", well, guess what happened right in that quarter, ending on June 25th 2016? AMD went back into profitability with a net income of 69 million. Not a lot, but considering the constant net loss in the previous years, a big win back to success.
The article also doesn't mention VR, not even once.

It's worth to note here where AMD actually made profit (although numbers for that quarter were only released one month later, on July 21st 2016). The official AMD press release for Q2 2016 states:

Enterprise, Embedded and Semi-Custom segment revenue of $592 million increased 59 percent sequentially and increased 5 percent year-over-year due to higher sales of semi-custom SoCs.

Note that the stock price of AMD was meanwhile at $5, yet the article suggested that the share will drop.

  -------------------------------------------

Meanwhile, at the time of this writing on September 2nd and 4th 2016, the share price is at $7.51. An increase of ~170% given the share price of $2.77 on January 4th 2016 and now the current share price. If you read and believed in these articles, you missed a great opportunity and a "safe" gainer.

AMD January till September 2016 

There were a lot more of these negative articles, I just picked some which I read at that time and which stuck in my head. Also Seeking Alpha, another financial analyst blog, had quite a lot of negative AMD news - however lthe articles from The Motley Fool caught my eye as they were almost always negative about AMD. Why's that? Actually after every article published on The Motley Fool, there's a disclaimer attached. Interestingly this shows, that:

The Motley Fool recommends Intel and Nvidia. 

And why do they recommend Intel and Nvidia? The disclaimer on an older article (the one I quoted from January 2014) shows:

Ashraf Eassa owns shares of Intel. The Motley Fool recommends Intel. The Motley Fool owns shares of Intel. 

So all these articles with a negative touch are purely motivated by The Motley Fool's own interests, instead of reporting independently and therefore being a "real" source of advise for investors. If the shares of AMD's competitors go up, The Motley Fool wins. Clever, to say at least.

Everybody who bet against these articles, follow his own gut and most importantly being able to count one and one together, me included, won. It's going to be interesting to see whether the articles will turn positive in case The Motley Fool suddenly owns shares of AMD themselves.

To finish this rant, let me do it, as these so-called fi-tech analysts are doing it:

I give AMD a strong buy, given the current bullish performing. In the short term (now to YTD) I expect AMD to reach $9. In the longterm, given VR hits the main markets and AMD continues with success in sales, technology and also reports more steady results (=profits), I expect AMD to hit $15 until or before 2018.
Don't take my word for it though, because I'm not a financial advisor/analyst ;-).

 

Slow down central mail relay server but with domain exceptions
Tuesday - Aug 30th 2016 - by - (0 comments)

I recently picked up a task to create a new central mail relay server in order to replace an existing relay system running on Ironport appliances. I decided to go with a setup of two Ubuntu 16.04 Xenial servers, using a VRRP-VIP for high availability and run Postfix as mail daemon.

Fun fact: That's kind of the opposite of what I had to do back in August 2011 (see Spam comparison: Ironport vs. Open Source solution), but that was for incoming and not for outgoing mails.

As this central mail relay sends a lot of mails for all internal servers, it's important that it does not get on a black list. And if it does, I need to be informed immediately and I need to keep the numbers of sent spam mails as low as possible.

In order to achieve that, I  decided to use throttling/rate limiting to send outgoing mails in a "controlled" way, so Postfix wouldn't just flush thousands of mails in a hurry.
This is what I added into /etc/postfix/main.cf:

# Throttling/Rate limiting
smtp_destination_concurrency_limit = 2
smtp_destination_rate_delay = 1s
smtp_extra_recipient_limit = 10

smtp_destination_concurrency_limit: As per the Postfix documentation: "The maximal number of parallel deliveries to the same destination via the smtp message delivery transport." Example: If your mail server sends mails to several gmail.com recipients, Postfix will only deliver two mails at the same time. The default is 20. This could also be set with default_destination_concurrency_limit.
This setting is particularly helpful in case a spammer bombards a certain domain. Often the recipient mail server then refuses the sent mails (your real mails as well) with a 421, 450 or 451 SMTP response that you're sending too many mails too quickly.

smtp_destination_rate_delay: Postfix doc says "The default amount of delay that is inserted between individual deliveries over the same message delivery transport, regardless of destination". In the setting above, I added a "wait time" of 1 second before the next e-mail should be delivered. The default is "0s" which means no delay between sending mails.
This setting slows down the sending a lot, although 1 second doesn't seem like a high value.

smtp_extra_recipient_limit: Again the doc: "This extra recipient space is reserved for the cases when the Postfix queue manager's scheduler preempts one message with another and suddenly needs some extra recipients slots for the chosen message in order to avoid performance degradation.". Understood? No, because for once the documentation doesn't describe it in plain English. A better explanation is this phrase: "Limit the number of recipients of each message. If a message had 20 recipients on the same domain, postfix will break it out to two different email messages instead of one." which comes from http://steam.io/2013/04/01/postfix-rate-limiting/.

This worked pretty well but once the system went live, I realized that too many mails queued up. As I mentioned before, this mail relay server is a central system for a lot of internal servers. Hence most of the e-mails are actually destined for internal employees (reports, statistics, infos, cronjobs, etc). Instead of piling these mails up in the queue, they can and should be delivered asap/without going through the throttling.

I decided to use transport maps to solve this. In /etc/postfix/main.cf I added the transport_maps parameter and defined where to find the (hashed) map:

# Use transport maps for special domain settings
transport_maps = hash:/etc/postfix/transport

In the transport file (/etc/postfix/transport) I defined the destination domains:

# cat /etc/postfix/transport
mycompany.ch fast:[external.relay.server]
internal.ch  fast:[external.relay.server]

Which means that every mail with a recipient address @mycompany.ch or @internal.ch will be sent through the "fast" Postfix service to the external relay server (external.relay.server). 

In order to create the hashed transport map, it needs to be created with postmap:

postmap /etc/postfix/transport

The "fast" service doesn't exist yet, so this needs to be created in /etc/postfix/master.cf:

# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (no)    (never) (100)
# ==========================================================================
[...]
# Special fast mail delivery for certain domains (specified in transport map)
fast      unix  -       -       n       -       -       smtp
        -o smtp_fallback_relay=

This tells Postfix to create a new service called fast. Additionally I added smtp_fallback_relay without a value. If the defined mail relay (external.relay.server) is unavailable, Postfix will attempt to send the mails directly via its own Internet connection.

In order to tell Postfix to use different throttling settings for this "fast" service, the following additional parameters were defined in /etc/postfix/main.cf:

# Faster rates for internal domains (specified in transport map)
fast_destination_concurrency_limit = 50
fast_destination_rate_delay = 0s
fast_extra_recipient_limit = 100

The fast_ prefix means that these are parameters for the "fast" transport service (compared to the smtp transport, which exists by default). As you can see, the values are much higher, letting through a lot of mails at the same time.

At the end to activate the config changes, a restart (not reload, because of the change in master.cf) of Postfix is required:

service postfix restart

Within a couple of minutes the mail queue went down from ~700 mails to 0, due to the amount of mails destined for internal domains/employees.

To sum it up: All e-mails which are sent to external domains (like hotmail.com, gmail.com, etc) are going through the slow throttling of this relay system. Yet when the destination is one of the defined internal domains, the mails are sent immediately and with a much higher rate.

 

rsyslog not logging into /var/log/mail.log? Check permissions!
Monday - Aug 22nd 2016 - by - (0 comments)

For a week or so I wondered why on one SMTP server (Ubuntu 16.04 Xenial with Rsyslogd and Postfix) rsyslog never logged into /var/log/mail.log, although this is clearly defined in the rsyslog config file:

 # cat /etc/rsyslog.d/50-default.conf | grep mail
mail.*                -/var/log/mail.log
# Logging for the mail system.  Split it up so that
#mail.info            -/var/log/mail.info
#mail.warn            -/var/log/mail.warn
mail.err            /var/log/mail.err
#    news.none;mail.none    -/var/log/debug
#    mail,news.none        -/var/log/messages
#daemon,mail.*;\
daemon.*;mail.*;\

Instead all log entries from the mail facility were logged into /var/log/syslog.

Yet on another SMTP server the mail facility log entries were correctly logged into /var/log/mail.log. Strangely enough, both systems were set up the same way.

Today I got some time for investigation and found out, that the permissions of the folder /var/log was different:

On SMTP01 (where mail logging happened into /var/log/syslog):

root@smtp01:/var# stat log
  File: 'log'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
Device: fc00h/64512d    Inode: 1005        Links: 11
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (  108/  syslog)
Access: 2016-08-22 08:29:56.243493060 +0200
Modify: 2016-08-22 08:29:55.747484499 +0200
Change: 2016-08-22 08:29:55.747484499 +0200
 Birth: -

On SMTP01 (where mail logging happened correctly into /var/log/mail.log):

 root@smtp02:/var# stat log
  File: 'log'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
Device: fc01h/64513d    Inode: 1005        Links: 11
Access: (0775/drwxrwxr-x)  Uid: (    0/    root)   Gid: (  108/  syslog)
Access: 2016-08-22 08:25:37.991669507 +0200
Modify: 2016-08-22 06:25:04.620044011 +0200
Change: 2016-08-22 06:25:04.620044011 +0200
 Birth: -

On SMTP01 the permissions were 0755, on SMTP02 0775. Big difference!

After I set the same permissions on smtp01 and restarting rsyslogd, logging of the mail facility started into /var/log/mail.log.

However I still don't know where this permission diff came from. In no logfile (and I have command auditing active) I was able to find a command who'd have edited the permissions.

 

Testing multicast is working with omping
Wednesday - Aug 10th 2016 - by - (0 comments)

Needed to see if multicast connections between two hosts were working. omping is designed to do this for you:

"The omping is program which uses User Datagram Protocol to determine if computer is able to send and/or receive IP unicast and multicast or Broadcast packets from the network."

In my case I needed to see if two CentOS 7 hosts are able to communicate with each other with multicast. On each host the program omping was installed:

yum install omping

And then omping is launched on both hosts:

omping host1 host2

 The output on both hosts should show the omping replies from the other host:

[root@host1 ~]# omping host1 host2
host2 : waiting for response msg
host2 : joined (S,G) = (*, 232.43.211.234), pinging
host2 :   unicast, seq=1, size=69 bytes, dist=0, time=0.122ms
host2 :   unicast, seq=2, size=69 bytes, dist=0, time=0.258ms
host2 : multicast, seq=2, size=69 bytes, dist=0, time=0.371ms
host2 :   unicast, seq=3, size=69 bytes, dist=0, time=0.328ms
host2 : multicast, seq=3, size=69 bytes, dist=0, time=0.434ms
host2 :   unicast, seq=4, size=69 bytes, dist=0, time=0.249ms
host2 : multicast, seq=4, size=69 bytes, dist=0, time=0.374ms
host2 :   unicast, seq=5, size=69 bytes, dist=0, time=0.270ms
host2 : multicast, seq=5, size=69 bytes, dist=0, time=0.379ms
host2 :   unicast, seq=6, size=69 bytes, dist=0, time=0.240ms
host2 : multicast, seq=6, size=69 bytes, dist=0, time=0.348ms
host2 :   unicast, seq=7, size=69 bytes, dist=0, time=0.272ms

[root@host2 ~]# omping host1 host2
host1 : waiting for response msg
host1 : waiting for response msg
host1 : joined (S,G) = (*, 232.43.211.234), pinging
host1 :   unicast, seq=1, size=69 bytes, dist=0, time=0.116ms
host1 : multicast, seq=1, size=69 bytes, dist=0, time=0.235ms
host1 :   unicast, seq=2, size=69 bytes, dist=0, time=0.219ms
host1 : multicast, seq=2, size=69 bytes, dist=0, time=0.329ms
host1 :   unicast, seq=3, size=69 bytes, dist=0, time=0.700ms
host1 : multicast, seq=3, size=69 bytes, dist=0, time=0.800ms
host1 :   unicast, seq=4, size=69 bytes, dist=0, time=0.249ms
host1 : multicast, seq=4, size=69 bytes, dist=0, time=0.360ms
host1 :   unicast, seq=5, size=69 bytes, dist=0, time=0.232ms
host1 : multicast, seq=5, size=69 bytes, dist=0, time=0.323ms
host1 :   unicast, seq=6, size=69 bytes, dist=0, time=0.291ms
host1 : multicast, seq=6, size=69 bytes, dist=0, time=0.377ms
host1 :   unicast, seq=7, size=69 bytes, dist=0, time=0.291ms
host1 : multicast, seq=7, size=69 bytes, dist=0, time=0.382ms

Multicast works between these two hosts running in the same data center.

Now I added a third host, running outside of the data center. Which means the traffic needs to go through a couple of routers and switches. Will multicast still work?

[root@host1 ~]# omping host1 host2 host3
host2 : waiting for response msg
host3 : waiting for response msg
host2 : waiting for response msg
host3 : waiting for response msg
host2 : joined (S,G) = (*, 232.43.211.234), pinging
host2 :   unicast, seq=1, size=69 bytes, dist=0, time=0.158ms
host2 : multicast, seq=1, size=69 bytes, dist=0, time=0.283ms
host3 : waiting for response msg
host2 :   unicast, seq=2, size=69 bytes, dist=0, time=0.199ms
host2 : multicast, seq=2, size=69 bytes, dist=0, time=0.323ms
host3 : joined (S,G) = (*, 232.43.211.234), pinging
host3 :   unicast, seq=1, size=69 bytes, dist=6, time=4.050ms
host2 :   unicast, seq=3, size=69 bytes, dist=0, time=0.267ms
host2 : multicast, seq=3, size=69 bytes, dist=0, time=0.382ms
host3 :   unicast, seq=2, size=69 bytes, dist=6, time=3.990ms
host2 :   unicast, seq=4, size=69 bytes, dist=0, time=0.299ms
host2 : multicast, seq=4, size=69 bytes, dist=0, time=0.308ms
host3 :   unicast, seq=3, size=69 bytes, dist=6, time=4.110ms

[root@host2 ~]# omping host1 host2 host3
host1 : waiting for response msg
host3 : waiting for response msg
host1 : joined (S,G) = (*, 232.43.211.234), pinging
host1 :   unicast, seq=1, size=69 bytes, dist=0, time=0.146ms
host1 :   unicast, seq=2, size=69 bytes, dist=0, time=0.229ms
host1 : multicast, seq=2, size=69 bytes, dist=0, time=0.338ms
host3 : waiting for response msg
host1 :   unicast, seq=3, size=69 bytes, dist=0, time=0.314ms
host1 : multicast, seq=3, size=69 bytes, dist=0, time=0.436ms
host3 : joined (S,G) = (*, 232.43.211.234), pinging
host3 :   unicast, seq=1, size=69 bytes, dist=6, time=4.108ms
host1 :   unicast, seq=4, size=69 bytes, dist=0, time=0.208ms
host1 : multicast, seq=4, size=69 bytes, dist=0, time=0.316ms
host3 :   unicast, seq=2, size=69 bytes, dist=6, time=3.981ms
host1 :   unicast, seq=5, size=69 bytes, dist=0, time=0.257ms

claudio@host3 ~ omping host1 host2 host3
host1 : waiting for response msg
host2 : waiting for response msg
host1 : joined (S,G) = (*, 232.43.211.234), pinging
host2 : joined (S,G) = (*, 232.43.211.234), pinging
host2 :   unicast, seq=1, size=69 bytes, dist=6, time=3.948ms
host1 :   unicast, seq=1, size=69 bytes, dist=6, time=3.985ms
host1 :   unicast, seq=2, size=69 bytes, dist=6, time=4.076ms
host2 :   unicast, seq=2, size=69 bytes, dist=6, time=4.072ms
host2 :   unicast, seq=3, size=69 bytes, dist=6, time=4.072ms
host1 :   unicast, seq=3, size=69 bytes, dist=6, time=4.112ms
host1 :   unicast, seq=4, size=69 bytes, dist=6, time=4.045ms
host2 :   unicast, seq=4, size=69 bytes, dist=6, time=4.056ms

 While multicast between host1 and host2 still works (as one can see in the replies), all connections to host3 only work using unicast. In this case multicast connectivity to host3 does not work.

 


Go to Homepage home
Linux Howtos how to's
Nagios Plugins nagios plugins
Links links

Valid HTML 4.01 Transitional
Valid CSS!
[Valid RSS]

7782 Days
until Death of Computers
Why?