I Debug, Therefore I Exist

Icon

Tag Line

Latency: A current perspective

Just a quick post on an interesting image I stumbled upon today. It really brings into perspective what the slowest part of a computer system is and where your optimizations should be focused on if you want efficiency. Since it’s a large, but thin image, I couldn’t thumbnail it, so you will find it in the link below.

Image: Latency

I would highly recommend you check out the blog post referenced in the image: What Your Computer Does While You Wait

I really like this quote from the article:

Most simple instructions on the Core 2 takeĀ one clock cycle to execute, henceĀ a third of a nanosecond at 3.0Ghz. For reference, light only travels ~4 inches (10 cm) in the time taken by a clock cycle.

Filed under: Programming

Varnish – Pretty fly for a reverse proxy

Hypothetical:

So, your website has been gaining traffic and the code has gotten more and more complex and things are starting to slow down. You’ve exhausted the most obvious tweaks, like optimizing a bit of your code (there’s only so much this can do to help), your database queries (a usual source of bottlenecks) and deployed a distributed cache (ie Memcache, but there are only so many things that you can cache effectively) and it’s alleviated things for a little bit, but it’s not really fixing the problem. You are just a sudden surge in traffic away from a dead web or database server. It’s no fun.

Sometimes, it’s not even the load on your servers that is the main problem. Maybe you already implemented a load balancer that is helping soften the load to each server, but you just aren’t able to effectively cache your pages, images, css, or javascript files (ie. each web server outputting their own etag’s, continuously invalidating the client browsers cache) and you are just wasting bandwidth, causing sluggish response times.

If any of this is familiar, you should look into Varnish. While there are some things that can be solved using PHP’s APC (or whatever alternative there is for other web languages) or even with your own cooked up solution, sometimes you just need to look at your assets from a structural point of view. Inevitably, you will need to consider having a reverse proxy.

I can certainly appreciate a certain apprehension towards using reverse proxies, because you are handing a level of control away from your application (your web servers), which just makes things a bit more complication than it already is, but you need to consider the benefits.

In the case of Varnish, it’s really fast. Really, really fast… One of the neat things about Varnish is what it offers you in terms of scripting. You can really pack a nice and simple, but powerful configuration into one file. If you want load balancing, Varnish will let you define your back-end servers and, if you want to get fancy, you can set up multiple back-end groups, if you want to apply different rules depending on whatever factors your define – url, http host, client IP, request method, etc. This can provide with a powerful way of architecting your services, even if it’s something as simple as blacklisting offending IP’s and directing them to a secondary server, which is low priority and doesn’t matter if it gets foobar’ed.

In terms of caching, you have to consider the two aspects of it:

  1. The most ideal situation for caching is having the user/client’s browser cache as much as possible and only expire when need be.
  2. Varnish will cache content to memory, if you want it to (Varnish will be conservative by default, though).

What this means is that a page (or item) will start by being requested from your web server, but will be cached in Varnish as well. If that same client requests the same resource, it will be cached in their browser, resulting in the fastest response time for the user and the least amount of bandwidth used by your servers. If another client requests the same resource, they will need to download it, but it will come directly from Varnish’s cache and never hitting your busy web servers. The benefits should be immediately obvious.

You need to work a bit on defining what pages are cached and how. You’ll probably want to put in rules for caching of certain types of files, if you want to take advantage of Varnish’s own cache. It requires a bit of thought and a bit of experimentation, but, on the flip side, it’s not all that complicated, once you understand the basics and there are only so many things to remember. My own configuration is nothing more than 20 or so lines of code.

One other very neat thing about Varnish is that you are able to switch configurations on the fly, by telnet’ing into Varnish’s admin console. Just load in the new configuration and tell Varnish to deploy it. This protects you from having to invalidate Varnish’s cache, by restarting it, and makes the switch seamless to your users.

I will write a follow up post on what configuration I’ve been using, since I have a slightly more complicated setup – I’ve also taken in account a few additional things, like what happens under certain conditions where pages cannot be served or Varnish is restarted and your back-end servers are yet to be recognized as healthy.

PS. Be sure to use as recent a version of Varnish, however. If you run Debian servers, look to use the testing packages.

Filed under: Caching

Calendar

July 2009
M T W T F S S
« Jun    
 12345
6789101112
13141516171819
20212223242526
2728293031  

Pages