Yesterday I hit a very strange error, when I attempted to run ntpdate on a Ubuntu 14.04 Desktop:
root@ubuntudesktop:~# ntpdate ntp.example.local
Error resolving ntp.example.local: Name or service not known (-2)
24 Nov 09:17:16 ntpdate[21746]: Can't find host ntp.example.local: Name or service not known (-2)
24 Nov 09:17:16 ntpdate[21746]: no servers can be used, exiting
The domain (ntp.example.local) could not be resolved.
Looks like a typical DNS resolving issue - at first sight. However DNS is working and resolving correctly:
root@ubuntudesktop:~# dig -t A ntp.example.local +short
dc001.example.local.
10.10.110.20
root@ubuntudesktop:~# nslookup ntp.example.local
[...]
ntp.example.local canonical name = dc001.example.local.
Name: dc001.example.local
Address: 10.10.110.20
The same command was tested on a Ubuntu 14.04 Server in the same subnet and there it worked without problems. What's the difference between the Desktop and Server version?
Strace gives more details. I will spare you the full output and show only the relevant output:
root@ubuntudesktop:~# strace ntpdate ntp.example.local
[...]
open("/etc/hosts", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=298, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f01da876000
read(3, "127.0.0.1\tlocalhost\n194.40.217.2"..., 4096) = 298
read(3, "", 4096) = 0
close(3) = 0
munmap(0x7f01da876000, 4096) = 0
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=87882, ...}) = 0
mmap(NULL, 87882, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f01da861000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libnss_mdns4_minimal.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\f\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=10432, ...}) = 0
mmap(NULL, 2105560, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f01d968a000
mprotect(0x7f01d968c000, 2093056, PROT_NONE) = 0
mmap(0x7f01d988b000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7f01d988b000
close(3) = 0
mprotect(0x7f01d988b000, 4096, PROT_READ) = 0
munmap(0x7f01da861000, 87882) = 0
socket(PF_LOCAL, SOCK_STREAM, 0) = 3
fcntl(3, F_GETFD) = 0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
connect(3, {sa_family=AF_LOCAL, sun_path="/var/run/avahi-daemon/socket"}, 110) = 0
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
fstat(3, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f01da876000
lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
write(3, "RESOLVE-HOSTNAME-IPV4 ntp.example."..., 38) = 38
read(3, "-15 Timeout reached\n", 4096) = 20
close(3) = 0
munmap(0x7f01da876000, 4096) = 0
write(2, "Error resolving ntp.example.local:"..., 64Error resolving ntp.example.local: Name or service not known (-2)
) = 64
[...]
On the Ubuntu Server, the strace output starts the same way, however there is a small difference:
root@ubuntuserver:~# strace ntpdate ntp.example.local
[...]
open("/etc/resolv.conf", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=242, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc831bc4000
read(3, "# Dynamic resolv.conf(5) file fo"..., 4096) = 242
read(3, "", 4096) = 0
close(3) = 0
munmap(0x7fc831bc4000, 4096) = 0
open("/etc/hosts", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=225, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc831bc4000
read(3, "127.0.0.1\tlocalhost\n194.40.217.1"..., 4096) = 225
read(3, "", 4096) = 0
close(3) = 0
munmap(0x7fc831bc4000, 4096) = 0
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=43388, ...}) = 0
mmap(NULL, 43388, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fc831bba000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libnss_dns.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\21\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=22952, ...}) = 0
mmap(NULL, 2117896, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc8309d5000
mprotect(0x7fc8309da000, 2093056, PROT_NONE) = 0
mmap(0x7fc830bd9000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4000) = 0x7fc830bd9000
close(3) = 0
mprotect(0x7fc8309d1000, 4096, PROT_READ) = 0
mprotect(0x7fc830bd9000, 4096, PROT_READ) = 0
munmap(0x7fc831bba000, 43388) = 0
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=242, ...}) = 0
open("/etc/resolv.conf", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=242, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc831bc4000
read(3, "# Dynamic resolv.conf(5) file fo"..., 4096) = 242
read(3, "", 4096) = 0
close(3) = 0
munmap(0x7fc831bc4000, 4096) = 0
socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.10.110.20")}, 16) = 0
[...]
As you can see in the strace output of the Server, the DNS name could be resolved to 10.10.110.20.
But where does the difference start? It's this line:
Desktop: open("/lib/x86_64-linux-gnu/libnss_mdns4_minimal.so.2", O_RDONLY|O_CLOEXEC) = 3
Server: open("/lib/x86_64-linux-gnu/libnss_dns.so.2", O_RDONLY|O_CLOEXEC) = 3
Ubuntu Desktop uses the DNS library libnss_mdns4_minimal.so, but Server uses the "normal" DNS library of libnss.
This behaviour is configured in /etc/nsswitch.conf which tells the Desktop version to use mdns4:
root@ubuntudesktop:~# cat /etc/nsswitch.conf | grep hosts
hosts: files mdns4_minimal [NOTFOUND=return] dns
Comparison with the Server:
root@ubuntuserver:~# cat /etc/nsswitch.conf | grep hosts
hosts: files dns
After removing the "mdns4_minimal [NOTFOUND=return]" entry from the hosts line in nsswitch.conf, ntpdate finally worked:
root@ubuntudesktop:~# ntpdate ntp.example.local
24 Nov 09:28:12 ntpdate[22745]: adjust time server 10.162.214.20 offset -0.008526 sec
No comments yet.
AWS Android Ansible Apache Apple Atlassian BSD Backup Bash Bluecoat CMS Chef Cloud Coding Consul Containers CouchDB DB DNS Database Databases Docker ELK Elasticsearch Filebeat FreeBSD Galera GlusterFS Grafana Graphics HAProxy HTML Hacks Hardware Icinga Icingaweb Icingaweb2 Influx Internet Java KVM Kibana Kodi Kubernetes LXC Linux Logstash Mac Macintosh Mail MariaDB Minio MongoDB Monitoring Multimedia MySQL NFS Nagios Network Nginx OSSEC OTRS Office PGSQL PHP Perl Personal PostgreSQL Postgres PowerDNS Proxmox Proxy Python Rancher Rant Redis Roundcube SSL Samba Seafile Security Shell SmartOS Solaris Surveillance Systemd TLS Tomcat Ubuntu Unix VMWare VMware Varnish Virtualization Windows Wireless Wordpress Wyse ZFS Zoneminder