HTTP content encoding and caching analysis with tcpdump and wireshark

Written by Claudio Kuenzler - 0 comments

Published on June 26th 2015 - Listed in Linux Network

To analyze a http caching problem I needed to grab the network connections and take a look into the http protocol and find possible problematic http requests. I used tcpdump to capture the tcp streams and wireshark to analyze the captured packets.

The following filters came in handy.

Show connections which requested www.example.com/ (the main domain) on the webserver 192.168.168.5:

http.request.uri == "/" && http.host == "www.example.com" && ip.dst == 192.168.168.5

Show connections which contain a HTTP 200 response code but don't contain the "Content-Encoding" http header:

http.response.code == 200 && !http.content_encoding

Show http responses where the content was gzip compressed:

http.content_encoding == gzip

Don't show http content, only headers (from http://www.askapache.com/hosting/debugging-http-cache-headers-wireshark.html):

http.response !=0 || http.request.method != "TRACE"

To be able to create filters with custom http headers, I first needed to add them to Wiresharks preferences:
Edit -> Preferences -> Protocols -> HTTP -> Custom HTTP headers fields -> Edit

I added the following additional headers:

Age: Age of Cache
X-Cache: Cache Type
X-Varnish-Hostname: Hostname of Varnish Cache

To use these custom http headers as a filter, you need to use the http.header prefix.

Show http responses which weren't cached, which don't contain the "Content-Encoding" header and which were treated by varnish-3 server:

http.header.X-Cache == MISS && !http.content_encoding && http.header.X-Varnish-Hostname == varnish-3

Show http responses which were cached (HIT) and treated by varnish-4 server:

http.header.X-Cache == HIT && http.header.X-Varnish-Hostname == varnish-4

Show http responses which pass through a varnish server (so the header X-Varnish-Hostname exists), have response code 200 and don't contain the "Content-Encoding" header:

http.header.X-Varnish-Hostname && http.response.code == 200 && !http.content_encoding