When I added some changes to a domain in our PowerDNS authoritative DNS server through Opera DNS UI, I was quite surprised that the changes were not automatically applied on the slave servers. As I wrote in a previous article, I had set up a DNS replication between the master and two slaves.
After further investigation I saw that the SOA serial was not increased when I changed the zone in the Opera DNS UI.
A notify was sent to the slaves, but due to the unchanged serial, no AXFR (DNS transfer) happened.
May 16 14:42:45 inf-dns01a-p pdns_server[23659]: Queued notification of domain 'example987.com' to 10.10.100.53:53
May 16 14:42:45 inf-dns01a-p pdns_server[23659]: Queued notification of domain 'example987.com' to 10.10.100.153:53
May 16 14:42:45 inf-dns01a-p pdns_server[23659]: Queued notification of domain 'example987.com' to 10.10.100.253:53
May 16 14:42:45 inf-dns01a-p dnsui: client_ip=192.168.88.19;uid=claudio;zone=example987.com.;object=zone;action=update;status=succeeded
May 16 14:42:45 inf-dns01a-p pdns_server[23659]: Received NOTIFY for example987.com from 10.10.100.53 but slave support is disabled in the configuration
May 16 14:42:46 inf-dns01a-p pdns_server[23659]: Received unsuccessful notification report for 'example987.com' from 10.10.100.53:53, error: Not Implemented
May 16 14:42:46 inf-dns01a-p pdns_server[23659]: Removed from notification list: 'example987.com' to 10.10.100.53:53 Not Implemented
May 16 14:42:46 inf-dns01a-p pdns_server[23659]: Removed from notification list: 'example987.com' to 10.10.100.153:53 (was acknowledged)
May 16 14:42:46 inf-dns01a-p pdns_server[23659]: Removed from notification list: 'example987.com' to 10.10.100.253:53 (was acknowledged)
Note: You can see that both slaves (10.10.100.153 and 10.10.100.253) have acknowledged the notify, but they did not initiate a AXFR.
When I first evaluated the user interface, being able to change DNS records and DNS slave replication was of course one of the main checks on my checklist. What changed?
It turns out that when a domain/zone is created in the DNS UI, a new database entry is added into the table "domainmetadata" in the PowerDNS database:
mysql> select * from domainmetadata;
+----+-----------+--------------+---------------------+
| id | domain_id | kind | content |
+----+-----------+--------------+---------------------+
| 2 | 1 | SOA-EDIT-API | INCEPTION-INCREMENT |
| 4 | 105 | SOA-EDIT-API | INCEPTION-INCREMENT |
| 6 | 139 | SOA-EDIT-API | INCEPTION-INCREMENT |
| 8 | 140 | SOA-EDIT-API | INCEPTION-INCREMENT |
| 10 | 141 | SOA-EDIT-API | INCEPTION-INCREMENT |
| 12 | 173 | SOA-EDIT-API | INCEPTION-INCREMENT |
+----+-----------+--------------+---------------------+
6 rows in set (0.00 sec)
Only a couple of entries exist, yet this authoritative DNS server holds more than 200 domains:
mysql> select count(*) from domains;
+----------+
| count(*) |
+----------+
| 209 |
+----------+
1 row in set (0.00 sec)
What's the difference between the domains added in domainmetadata and the other domains which didn't get an entry? All the domains which appear in the domainmetadata table were added manually in the DNS UI. All the other domains were created through a script which uses the pdnsutil command line.
I manually inserted a domain into domainmetadata:
mysql> insert into domainmetadata (domain_id, kind, content) VALUES (214, 'SOA-EDIT-API', 'INCEPTION-INCREMENT');
Query OK, 1 row affected (0.01 sec)
mysql> select * from domainmetadata;
+----+-----------+--------------+---------------------+
| id | domain_id | kind | content |
+----+-----------+--------------+---------------------+
| 2 | 1 | SOA-EDIT-API | INCEPTION-INCREMENT |
| 4 | 105 | SOA-EDIT-API | INCEPTION-INCREMENT |
| 6 | 139 | SOA-EDIT-API | INCEPTION-INCREMENT |
| 8 | 140 | SOA-EDIT-API | INCEPTION-INCREMENT |
| 10 | 141 | SOA-EDIT-API | INCEPTION-INCREMENT |
| 12 | 173 | SOA-EDIT-API | INCEPTION-INCREMENT |
| 13 | 214 | SOA-EDIT-API | INCEPTION-INCREMENT |
+----+-----------+--------------+---------------------+
7 rows in set (0.00 sec)
And then did some changes through the DNS UI for that domain with domain id 214. Result: It worked! The SOA serial finally increased and the slaves launched their AXFR commands.
Updating domains/zones through PowerDNS's API requires an entry in the PowerDNS table domainmetadata, for each domain!
I will now adapt my Infoblox2PDNS migration script to create this table entry when creating a new domain through the cli.
The following bash one-liner will set the SOA-EDIT-API to INCEPTION-INCREMENT for every domain found in the domains table which has no SOA-EDIT-API in the domainmetadata table yet. Assuming here, the database name is "powerdns":
# for id in $(mysql -Bse "select id from powerdns.domains"); do exists=$(mysql -Bse "select count(domain_id) from powerdns.domainmetadata where domain_id = $id and kind = 'SOA-EDIT-API'"); if [[ $exists -gt 0 ]]; then echo "There is already a SOA-EDIT-API setting for this domain id $id, ignoring"; else echo "Setting SOA-EDIT-API to INCEPTION-INCREMENT for domain id $id"; mysql -e "insert into powerdns.domainmetadata (domain_id, kind, content) values ($id, 'SOA-EDIT-API', 'INCEPTION-INCREMENT')"; fi; done
[...]
Setting SOA-EDIT-API to INCEPTION-INCREMENT for domain id 203
Setting SOA-EDIT-API to INCEPTION-INCREMENT for domain id 205
Setting SOA-EDIT-API to INCEPTION-INCREMENT for domain id 206
Setting SOA-EDIT-API to INCEPTION-INCREMENT for domain id 207
Setting SOA-EDIT-API to INCEPTION-INCREMENT for domain id 208
Setting SOA-EDIT-API to INCEPTION-INCREMENT for domain id 209
Setting SOA-EDIT-API to INCEPTION-INCREMENT for domain id 210
Setting SOA-EDIT-API to INCEPTION-INCREMENT for domain id 211
Setting SOA-EDIT-API to INCEPTION-INCREMENT for domain id 212
Setting SOA-EDIT-API to INCEPTION-INCREMENT for domain id 213
There is already a SOA-EDIT-API setting for this domain id 140, ignoring
There is already a SOA-EDIT-API setting for this domain id 141, ignoring
Setting SOA-EDIT-API to INCEPTION-INCREMENT for domain id 31
[...]
Steve from Auckland wrote on May 11th, 2020:
You can also set this by using pdnsutil set-meta:
pdnsutil set-meta zonename SOA-EDIT-API INCEPTION-INCREMENT
... which doesn't require using the database.
This annoying feature wasted about 5 hours of my time today.
Daniel Lo Nigro from Palo Alto, CA wrote on Jul 10th, 2019:
Your bash one-liner could likely just be a single SQL INSERT INTO ... SELECT FROM query, but I guess the extra debugging output is useful :)
AWS Android Ansible Apache Apple Atlassian BSD Backup Bash Bluecoat CMS Chef Cloud Coding Consul Containers CouchDB DB DNS Database Databases Docker ELK Elasticsearch Filebeat FreeBSD Galera Git GlusterFS Grafana Graphics HAProxy HTML Hacks Hardware Icinga Icingaweb Icingaweb2 Influx Internet Java KVM Kibana Kodi Kubernetes LVM LXC Linux Logstash Mac Macintosh Mail MariaDB Minio MongoDB Monitoring Multimedia MySQL NFS Nagios Network Nginx OSSEC OTRS Office PGSQL PHP Perl Personal PostgreSQL Postgres PowerDNS Proxmox Proxy Python Rancher Rant Redis Roundcube SSL Samba Seafile Security Shell SmartOS Solaris Surveillance Systemd TLS Tomcat Ubuntu Unix VMWare VMware Varnish Virtualization Windows Wireless Wordpress Wyse ZFS Zoneminder