In my previous post I described an error in the installation of modified eCommerce shop which appeared during step 5 of the web installer (_installer).
I posted a workaround which helps to continue the installation until the end but it doesn't explain the actual source or reason why this error appears.
After further investigation, the error seems to be caused by... IPv6! What? Yes!
The function I previously wrote about (phpLinkCheck) uses fsockopen to create a tcp connection to the localhost on port 80:
$fp = @fsockopen($url["host"], $url["port"], $errno, $errstr, 30);
This handle is then used to check for the status code of robots.txt, depending on the path and hostname:
$httpRequest = "HEAD ". $url["path"] ." HTTP/1.1\r\n"
."Host: ". $url["host"] ."\r\n"
So if I simulate that tcp connection with telnet it looks like this:
# telnet www.example.com 80
Connected to www.example.com.
Escape character is '^]'.
GET /shop/robots.txt HTTP/1.1
HTTP/1.1 404 Not Found
Once I saw that output, I knew. Something is telling telnet to use IPv6 when accessing port 80 even though this website is on the localhost. In this case the Apache webserver is listening on all interfaces, however the virtual hosts are only listening on one IPv4 address. So if I connect to the IPv6 address and asking for the virtual host "www.example.com", Apache looks up its config and doesn't find any entry matching the IPv6 address and the domain. This results in the 404 return code.
So there are two solutions:
1) The web server must be configured, that all vhosts also listen to the IPv6 address. Depending on the architecture of the web server this can be possible. But in reality I've seen many systems where Apache listens on all ip addresses but the vhosts are configured only for IPv4.
2) The way of "checking for the availability of robots.txt" must change in the install_step5.php. It is also possible to first make a DNS lookup of the domain where the IPv4 A-entry is used in fsockopen.