How to set up log forwarding in KeyCDN and log events to a dedicated log file with Rsyslog

Written by - 0 comments

Published on - last updated on June 17th 2022 - Listed in Linux Cloud Network Internet

Content Delivery Networks (CDNs) are helpful for caching and delivering content across the globe, lowering your own bandwidth for assets (such as images or videos). KeyCDN is such a CDN provider.

There is one slight problem though: When you need to troubleshoot connections to a CDN provider you (mostly) can't see any access logs as you would on your own web server. But luckily KeyCDN (and maybe also others) provides a log forwarding using the syslog protocol. And with a proper rsyslog configuration, the KeyCDN logs can be written into a dedicated log file.

Log forwarding in KeyCDN

The setup part in the KeyCDN account is pretty straightforward. There's not much you can do wrong. In the navigation, select Account -> Log forwarding and you're greeted with a basic "Log forwarding" form.

Configure log forwarding in KeyCDN

Enter your own syslog server's FQDN or IP and the syslog server's listener port (default 514). Note that only UDP is supported.

Configure syslog server (Rsyslog)

First you need to make sure that Rsyslog accepts logs from remote servers and actually listens on a port. This is (usually) configured in /etc/rsyslog.conf by enabling the "imudp" (UDP input module):

# provides UDP syslog reception
input(type="imudp" port="514")

Another port can also be used, of course. But don't forget to update the log forwarding settings in KeyCDN then.

When the logs arrive on the syslog server (running with Rsyslog 8.2001 on Ubuntu 20.04), they should be written into a dedicated log file (/var/log/keycdn.log) instead of being mixed up with all kinds of other log events in /var/log/syslog.

To achieve this, a small configuration snippet needs to be added in /etc/rsyslog.d:

root@syslog:~# cat /etc/rsyslog.d/10-keycdn.conf
if $hostname == 'keycdn' then {
  action(type="omfile" File="/var/log/keycdn.log")

To understand this snippet better:

  • $hostname == 'keycdn': Interestingly the logs from KeyCDN are shipped with the hostname field being just "keycdn". There are no IPs or FQDNs shown in the raw log message. As the logs can be received from all kinds of KeyCDN servers with different IPs and in different network ranges, it makes sense to check for the "hostname" and match it with the "keycdn" value
  • action: If the condition matches, then rsyslog is told to do an action. In this case it should use the "file output module" (omfile) and use the file path "/var/log/keycdn.log"
  • stop: Rsyslog is then told to stop processing these logs. Without stop the logs would continue to be processed by (further) rsyslog configs

It's important to have this configuration snippet load BEFORE the default rsyslog rules (/etc/rsyslog.d/50-default.conf), otherwise you will see the KeyCDN logs appear in both /var/log/keycdn.log and /var/log/syslog. By using a lower number (10-keycdn.conf), this config is loaded first.

Restart Rsyslog after adding /etc/rsyslog/10-keycdn.conf:

root@syslog:~# systemctl restart rsyslog

Now the logs should appear in the dedicated log file /var/log/keycdn.log:

root@syslog:~# tail -n 2 /var/log/keycdn.log
Jun 17 10:20:47 keycdn  1655454048.016|defr|2003:c8:0:0:0:0:0:0|200|1203|uid18513|zid43124||HIT|"GET /js/9.10.3/app/workbox.4c4f5ca6.js HTTP/2.0"|[17/Jun/2022:08:20:48 +0000]|"-"|"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:101.0) Gecko/20100101 Firefox/101.0"|https|TLSv1.3|DE|NW|"AS3320 Deutsche Telekom AG"|-
Jun 17 10:20:47 keycdn  1655454048.017|defr|2001:16b8:0:0:0:0:0:0|200|1116|uid18513|zid43124||HIT|"GET /img/10.10.5/static/favicon/favicon-32x32.png?v=3 HTTP/2.0"|[17/Jun/2022:08:20:48 +0000]|""|"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.124 Safari/537.36 Edg/102.0.1245.41"|https|TLSv1.3|DE|BW|"AS8881 1&1 Versatel Deutschland GmbH"|-

The KeyCDN log format is explained here. Note that the client IP is anonymized by setting the last octet to "0" to be GDPR compliant. This applies to both IPv4 and IPv6 client addresses.

What about firewall restrictions?

That's a good question. The syslog server could be bombarded with data from the Internet. For security (and performance) reasons, the syslog listening port should only be opened to known sources. Unfortunately KeyCDN does not publicly announce their IP ranges. From the FAQ:

We don't disclose the IPs of the edge servers because the IPs frequently change.

However the (known) IP ranges belonging to KeyCDN's parent company (proinity LLC, AS44239) can be found on ipinfo.

By looking at the firewall logs, these IP ranges (so far) match the sending KeyCDN servers.

Update: After a quick discussion with the KeyCDN support, the following two ranges (as of June 2022) are relevant for syslog logging:


This is nicely documented in the logging-prefixes json.

Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.