A couple of weeks ago I thought that I had hit a strange bug within HAProxy.
The defined server of a backend was called with a domain like this:
server back1 backend.example.com:443 check ssl verify none
The domain backend.example.com itself points to a VIP, so high avaibility is in this case not managed by HAProxy but rather by the backend itself.
When I renewed the whole backend cluster and pointed the domain to the new VIP, I became aware that the whole traffic still runs through the old VIP - although DNS resolving on the HAProxy machine pointed the domain correctly to the new VIP.
Another systems engineer and I suspected a glitch in the DNS caching of that particular system, because after reloading HAProxy, the traffic went correctly towards the new VIP.
A follow-up on this topic with the same systems engineer turns out that this is "by design" in current HAProxy versions. In the announcement for HAProxy 1.6, the following new feature was mentioned:
Server IP resolution using DNS at runtime
In 1.5 and before, HAProxy performed DNS resolution when parsing configuration, in a synchronous mode and using the glibc (hence /etc/resolv.conf file).
Now, HAProxy can perform DNS resolution at runtime, in an asynchronous way and update server IP on the fly.
So only starting with HAProxy 1.6 the DNS lookup is done dynamically. As we probably all thought it would already do it.