Code Emitter

Till today I used Lighttpd to distribute the load to multiple FastCGI backends. This worked “ok” until now but I’m really impressed about the performance gained by adding Varnish 2.0 (a high performance HTTP reverse proxy) in front of the web cluster. Additionally this is much easier than always fixing the FastCGI stuff when it is broken again.

However, Varnish 2.0 was released just 2 days ago, so there are still some issues. It seems that the random director (directors are used to distribute the load to multiple backend webservers) is broken currently but round-robin is working fine and should last for a while. As a workaround for using the random .weight option you can just add a backend more than once to the round-robin director.

There was also a problem with the varnishstat Munin plugin at Munin Exchange, so I uploaded a fixed version here and also added some installation instructions.

Do you wonder why Varnish seems not to cache anything?

For me (in fact I think for most people) the default varnish configuration is a bit too restrictive when it comes to cookies. All requests that contain a “Cookie” request header will not be cached – never.  As soon as a cookie is set by the site, also all static files like images, scripts and styles will no longer be cached cause the cookie is sent along with every client request, even to static files. In most situations (for almost every site that requires a login or uses adsense or similar and sets a cookie) this will render Varnish absolutely useless. A better approach is to let the dynamic pages always set a cookie, so that a “Set-Cookie” response header is created every time. Varnish also will not cache when a “Set-Cookie” response header is present so we don’t need to care about the “Cookie” request header anymore. Knowing this, all we need to do is make Varnish ignore the cookies for static files – and this is easy:

# in vcl_recv
if (req.url ~ “\.(png|gif|jpg|swf|css|js)$”) {
unset req.http.Cookie;
}

This should work for 99% of all somewhat modern websites where users can log in. Because of this I think the documentation should mention this more clearly. Also don’t forget to set etag.use-inode = “disable” in lighttpd.conf to sync the ETags when using multiple backend servers. Anyhow, Varnish is great, so have fun! :)