You appear to be offline. Some site functionality may not work.

Normalizing the Host Header

Dec 15, 2014 in Varnish,Performance

In the continued quest to increase cache hit ratios, the chant is: "Normalize, normalize, normalize." Less variation in your requests means you have a higher chance of getting hits. This month's highlight is the Host header.

The default VCL that comes with Varnish can safely be put in front of a web server with multiple virtual hosts, since the Host header is used to determine the hash of each object. Because of this, https://www.example.com/main.html and https://example.com/main.html will be two separate objects in the cache.

Sometimes caching different objects on different hostnames is what you want. One could be the actual page, and the other could be a redirect to it. The redirect makes sure your URLs are the same everywhere and tends to look tidy. On the other hand, it could also cause a slight delay in the page load due to the possible redirect, so you might want all variations of hostnames in the URL to get the same response. Of course, you'd want all variations of a URL to use just one object in your cache.

This is incredibly simple to do:

sub vcl_recv {
  set req.http.Host = "example.com";
}

Maybe you also have assets on css.example.com and img.example.com, for which you have separate virtual hosts for on your webserver. In that case, you can extend your VCL with a condition:

sub vcl_recv {
  if (req.http.Host == "www.example.com") {
    set req.http.Host = "example.com";
  }
}

Now only www.example.com is changed to example.com and the other hostnames are left as they are.

A different use case might be that you have different language versions of your website, at nl.example.com, en.example.com, and fr.example.com. And the image, JavaScript, and CSS files for all these sites are all the same.

In this case you only rewrite the Host: header for those files, like so:

sub vcl_recv {
  # Language independent files
  if (req.url ~ "^/(img|css|js)/") {
    set req.http.Host = "nl.example.com";
  }
}

If you're feeling really fancy, you could replace nl.example.com with static.example.com. Whatever works best with your web server setup.

If there are images or JavaScript files that do have language dependent parts to them, you could simply place those in separate subdirectories from the root, like /nl/css/ or /en/js/.

As an aside, if you do use multiple hostnames for the same content, remember to use canonical links to help search engines determine which is the preferred hostname. See this canonical URL article for more information, and this blog post for pitfalls to avoid.

Varnish Performance

You may also like:

Subscribe to our newsletter

Subscribe to our newsletter

Analyze your origin logs to get a higher cache efficiency

If you want to increase the efficiency of your Varnish (or Fastly) cache, you need to figure out what traffic is not cached. By definition, any traffic that reaches your origin is not cached, and…

Accelerating Rails, Part 2: Dynamic HTTP Caching

In the second part of our series on accelerating Rails, I’ll cover configuration of a few Fastly features, Varnish and Varnish Configuration Language (VCL), and strategies for caching dynamic content that are targeted towards the…

Accelerating Rails, Part 1: Built-in Caching

Caching is one strategy that helps ease scaling pains that I often see Rails developers overlooking. Starting out with caching can be confusing, because terms and documentation can be convoluted, especially if you’re not an…

Author

Rogier Mulhuijzen | Senior Professional Services Engineer

Rogier “Doc” Mulhuijzen is a senior professional services engineer and Varnish wizard at Fastly, where performance tuning and troubleshooting have formed the foundation of his 18-year career. When he’s not helping customers, he sits on the Varnish Governance Board, where he helps give direction and solve issues for the Varnish open source project. In his spare time, he likes to conquer all terrains by riding motorcycles, snowboarding, and sailing.

drwilco