I Debug, Therefore I Exist


Tag Line

Latency: A current perspective

Just a quick post on an interesting image I stumbled upon today. It really brings into perspective what the slowest part of a computer system is and where your optimizations should be focused on if you want efficiency. Since it’s a large, but thin image, I couldn’t thumbnail it, so you will find it in the link below.

Image: Latency

I would highly recommend you check out the blog post referenced in the image: What Your Computer Does While You Wait

I really like this quote from the article:

Most simple instructions on the Core 2 take one clock cycle to execute, hence a third of a nanosecond at 3.0Ghz. For reference, light only travels ~4 inches (10 cm) in the time taken by a clock cycle.


Filed under: Programming

Varnish – Pretty fly for a reverse proxy


So, your website has been gaining traffic and the code has gotten more and more complex and things are starting to slow down. You’ve exhausted the most obvious tweaks, like optimizing a bit of your code (there’s only so much this can do to help), your database queries (a usual source of bottlenecks) and deployed a distributed cache (ie Memcache, but there are only so many things that you can cache effectively) and it’s alleviated things for a little bit, but it’s not really fixing the problem. You are just a sudden surge in traffic away from a dead web or database server. It’s no fun.

Sometimes, it’s not even the load on your servers that is the main problem. Maybe you already implemented a load balancer that is helping soften the load to each server, but you just aren’t able to effectively cache your pages, images, css, or javascript files (ie. each web server outputting their own etag’s, continuously invalidating the client browsers cache) and you are just wasting bandwidth, causing sluggish response times.

If any of this is familiar, you should look into Varnish. While there are some things that can be solved using PHP’s APC (or whatever alternative there is for other web languages) or even with your own cooked up solution, sometimes you just need to look at your assets from a structural point of view. Inevitably, you will need to consider having a reverse proxy.

I can certainly appreciate a certain apprehension towards using reverse proxies, because you are handing a level of control away from your application (your web servers), which just makes things a bit more complication than it already is, but you need to consider the benefits.

In the case of Varnish, it’s really fast. Really, really fast… One of the neat things about Varnish is what it offers you in terms of scripting. You can really pack a nice and simple, but powerful configuration into one file. If you want load balancing, Varnish will let you define your back-end servers and, if you want to get fancy, you can set up multiple back-end groups, if you want to apply different rules depending on whatever factors your define – url, http host, client IP, request method, etc. This can provide with a powerful way of architecting your services, even if it’s something as simple as blacklisting offending IP’s and directing them to a secondary server, which is low priority and doesn’t matter if it gets foobar’ed.

In terms of caching, you have to consider the two aspects of it:

  1. The most ideal situation for caching is having the user/client’s browser cache as much as possible and only expire when need be.
  2. Varnish will cache content to memory, if you want it to (Varnish will be conservative by default, though).

What this means is that a page (or item) will start by being requested from your web server, but will be cached in Varnish as well. If that same client requests the same resource, it will be cached in their browser, resulting in the fastest response time for the user and the least amount of bandwidth used by your servers. If another client requests the same resource, they will need to download it, but it will come directly from Varnish’s cache and never hitting your busy web servers. The benefits should be immediately obvious.

You need to work a bit on defining what pages are cached and how. You’ll probably want to put in rules for caching of certain types of files, if you want to take advantage of Varnish’s own cache. It requires a bit of thought and a bit of experimentation, but, on the flip side, it’s not all that complicated, once you understand the basics and there are only so many things to remember. My own configuration is nothing more than 20 or so lines of code.

One other very neat thing about Varnish is that you are able to switch configurations on the fly, by telnet’ing into Varnish’s admin console. Just load in the new configuration and tell Varnish to deploy it. This protects you from having to invalidate Varnish’s cache, by restarting it, and makes the switch seamless to your users.

I will write a follow up post on what configuration I’ve been using, since I have a slightly more complicated setup – I’ve also taken in account a few additional things, like what happens under certain conditions where pages cannot be served or Varnish is restarted and your back-end servers are yet to be recognized as healthy.

PS. Be sure to use as recent a version of Varnish, however. If you run Debian servers, look to use the testing packages.

Filed under: Caching

Dell PowerEdge + Debian + LVM + Encryption: How to get this combination to work

Since, at my company, we have built and are scaling our architecture, we often have to add and reformat servers. In addition, we use Debian pretty much exclusively. Generally, I am quite happy with the quality of Dell’s PowerEdge series. I won’t speak to their desktop workstations, but their servers are quite powerful, stable (haven’t had a failure as of yet, knock on wood), and come with a lot of nice extra features.

Debian is not an officially supported OS (and I don’t see why not). It’s either Windows, RedHat Linux or SUSE Linux. While RedHat Linux (including CentOS) and Novells SUSE Linux Enterprise Server are respectable server operating systems, we’ve settled on Debian because we’ve generally had a good experience with it and the community support for Debian far outweighs any other Linux distro that I’m aware of. Good community support means faster access to knowledge and fixes. That is just a fact of life and I see that, moving forward, this will make it harder for so-called enterprise products to compete with their limited resources. On the other hand, the downside to this is that you need an active IT staff, willing to commit time to managing quirks and solving problems on their own, without having any corporate support. This works well for us. We like more control.

Nevertheless, this post isn’t about the virtues of running one server OS over another. This is a post about a problem that has plagued us for a little while. Generally, Debian has an astounding support for all sorts of hardware, including server hardware, but occasionally there will be something that doesn’t quite work the way you want it to. In our case, it was using Debian (Etch & Lenny) on Dell PowerEdge 1950/2950 servers with the integrated PERC 5/i raid controller. The specific problem is that during the install process (using the netinstall cd, although I believe it should apply to all other install methods), the installer gets the SCSI drive assignment order all wrong, resulting in an un-bootable operating system once the install is complete and you reboot it.

If you are simply installing it without LVM, this should not cause too much of a headache. You just change the root drive in grub, boot it up, change /etc/fstab & /boot/grub/menu.1st and reboot. But the problem comes when you use LVM, and especially, as in our case, LVM + Luks encryption. Doing so will drop your system to BusyBox, with a system that would not boot and requires some massaging to get working. So, this post describes our solution, so that if anyone else hits this problem with their PERC 5/i raid controller, this should hopefully help them.

First things first, I will assume you have installed Debian Lenny at this point with LVM + encryption (in our case, the installer installed the boot partition into /dev/sdb1 and the encrypted volume group into /dev/sdb2) and you have rebooted only to be dumped to BusyBox with an error saying that such and such volume group could not be found. The next thing you need to do is unlock your encrypted drive.

I shall also assume that /dev/sda1 is your plain ext2 boot partition, while /dev/sda2 is your encrypted partition at this point.

The next thing you need to do is run:

# cryptsetup luksOpen /dev/sda2 sda2_crypt

This will ask you for your encrypted password and will then unlock the partition.

The next thing you need to do is initialize the LVM volumes:

# lvm

lvm> vgchange -a y <VOLUME-GROUP>

<VOLUME-GROUP> is the name of the volume group on your system. If you are unsure what it is, run “vgs” and it should list it. Usually it is something akin to your hostname, so if you hostname is “SRV1”, it would likely also be “SRV1”.

Now, all you have to do is exit the lvm console with the command “exit“.

Everything should be set up correctly now to continue your boot from BusyBox, so type “exit” again. Your system should resume booting. Don’t worry about any errors you get. You might get a warning about not being about to mount your boot partition and to enter your password to enter maintenance mode. This shouldn’t be necessary, though.  Just press CONTROL+D on your keyboard to continue.

Finally, you should hit a normal login prompt. Login as your root user. Now we will be editing a few files to adjust everything.

First, edit “/etc/fstab“:


/dev/sdb1       /boot           ext2    defaults        0       2


/dev/sda1       /boot           ext2    defaults        0       2

Next up, you need to edit “/etc/crypttab“:


sdb2_crypt /dev/sdb2 none luks


sda2_crypt /dev/sda2 none luks

Finally, we need to edit “/boot/grub/device.map“, but the /boot partition isn’t mounted yet, so type:

# mount -t ext2 /dev/sda1 /boot

Now edit the file “/boot/grub/device.map“:


(hd0)    /dev/sdb


(hd0)    /dev/sda

Finally, that’s it for editing things. You just need to run a final command to update your boot image. Run this:

# update-initramfs -u

That’s it, really. Now you can reboot your system and see how it should finally pick everything up automagically, prompt you for your encryption password and boot into Debian without any headaches.


If anyone wants a more in-depth explanation as to what is going on and why, leave me a comment and I’ll be happy to fill you in on the details.

Please note, you might have to repeat this process again if APT ever upgrades your kernel, so be aware of this.

Filed under: Uncategorized

New Internet Explorer Exploit

It was reported on the 11th of February on Heise-Online that a new exploit was found for Windows users running any version of Internet Explorer.

As usual, the nature of the exploit is related to Microsoft trying to soften web standards, in order to make the web a “friendlier” place for everyone. I wish someone would explain to Microsoft that the job of a web browser is to take a bunch of semantic code and render it. It’s not supposed to think on behalf of the user or the developer. I suspect the decision (way back when) to add this “feature” was made by non-tech literate managers, to whom an image is an image (“Who cares about formats! I don’t want to see web pages break or act funny… just make it work the way I want it to“). Ultimately, poor decisions coming from a company as predominant to the lives of most people as Microsoft is – a company, who knew that releasing a web browser by default on their operating system, would amount to an instant grab of 80-90% of the browser market (although, fortunately, this is no longer the case) – causes more headaches for everyone in the long run.

At the end of the day, this makes life more difficult for:

  • developers having to apply fixes in their increasingly hacked code and feel constantly paranoid
  • users end up having more of their systems compromised by clever black hats (that’s what the rest of you would call ‘hackers’, although this is a false use of the term)
  • and for Microsoft, who ends up having to layer patch on top of patch – not only to fix the original problem, but to fix the mess it created (meaning an endless supply of patches, which users eventually will ignore, because they see them so often that they become apathetic).

Ok, well, rant aside… this new exploit has to do with images. If you are using Internet Explorer (and if you are, you should really consider downloading Firefox or some other alternative instead), this exploit allows hackers to inject JavaScript into images, which is then run on your system in your browser.

This may seem harmless to you, but consider this: If a malicious coder uploads a profile image to some forum or some other site, which does not filter the image (ie. run it through ImageMagick or some other image processor), then they can embed JavaScript into the image. When you load the page, Internet Explorer then loads in all the elements on the page (HTML, CSS, JavaScript, Flash, etc) and when IE loads in one of these nasty images, IE instead runs the code in them. If you are still wondering why this is bad? Because this allows these malicious users to control your browser in some unexpected ways, such as giving them the ability to steal your cookies. If you are yet still wondering why this is a bad thing: this gives those malicious users the ability to login into your account, view your personal information, change it, steal it, pretend to be you or even lock you out of your own account.

I hope you realize by now how bad this exploit can be.

I tested this exploit on Microsoft Internet Explorer 6 & 7, Mozilla Firefox 3.0, Opera 9.6 and Google Chrome. I can confirm that this exploit affects only Internet Explorer 6 & 7. None of the others had this problem. I haven’t tested IE 8, but I would be surprised if it were any different until Microsoft decides to patch it (if they ever do).

As always, if you want to stay secure:

  1. Don’t use Internet Explorer. Install Firefox instead.
  2. Install NoScript. It may make your internet surfing a bit more painful at times, but the security implications are far more worth it than the annoyance it can cause. Remember, you always have to compromise between ease-of-use and security. You can’t have your cake and eat it too.
  3. If you absolutely have to run any version of Windows, instead of the more secure alternatives (Apple Mac OS X, Ubuntu or other flavors of Linux), make sure you keep your system up-to-date as soon as you are notified of something. Don’t think you just need to keep your operating system up-to-date. You need to make sure your other software is also up-to-date as well.
  4. Make sure you run your computer behind a router (NAT router, specifically. Good home routers are quite cheap these days. Even those under $50 would be sufficient). This protects you against remote execution vulnerabilities, such as the recent Conficker, because your computer will be hidden behind a router, which does not provide any direct route for people to access your computer unless it somehow expects it. This shields you largely form malicious people scanning the internet for vulnerable computers. If your computer is connected directly to your internet modem, you are asking for trouble.
  5. Maintain a “Trust No One” frame of mind, while connected to the internet. Don’t trust anyone or any site unless you can establish a reasonable degree of trust.
  6. Some of you will say, “What about a firewall or anti-virus?” If you are a Windows user, the unfortunate reality is that an anti-virus application is absolutley necessary, so give AVG or ClamWin a go. They are both free (although AVG is only free for personal use). I recommend you install both. And, as for firewalls: it can’t hurt to have a firewall installed, but, ultimately, if you follow steps 1-5 above, it will be highly unlikely you will ever need one, unless you run a big network with lots of computers and users. If you are maintaining your own home network with few computers, then there isn’t too much benefit.

Well, I hope you got something out of this security alert.

If you have any questions, just leave a comment. Good luck.

Filed under: Security

Dino*Run: Escape Extinction

I have to say, I’m usually not a big fan of flash games, but this one, I have to say, impressed me a great deal.

You are a dinosaur in a retro, pixelated setting, trying to escape doom from above. You run through the wilderness trying to avoid obstacles that would slow you down, devouring snacks on the way and climbing over other dinosaurs to reach your goal.


That’s as much as I will say about it, because I don’t want to spoil the surprise. I only want to say this: your heart will be pounding quite a few times. And that’s no small feat for a Flash game.

The music is also very retro and everything in the game is actually very visually appealing – especially if you like the style of Another World and Flashback. The environment has a lot of vivid detail and, while the controls aren’t perfect, it feels authentic to the retro-style of the game.

In any event, I’m sure after you give it a go, you’ll agree that it’s one of the finer mini games out there (lots of unlockable goodies!) and you’ll keep coming back for another run.

Go play it!

Filed under: Uncategorized, , , ,

Miss the old Mac rounded corners? Displaperture is for you

I have to admit, while I didn’t use to be a Mac user and spent a good deal of time criticizing it in the way a Windows user usually does, I always had a deep, hidden admiration for the UI. ‘Course, now I’m out of the closet (since getting a MacBook a year ago), so now I can admit things freely.

One thing that kind of disappointed me when I first booted OS X on my fresh new MacBook was the missing rounded corners. I don’t know why, but it gives the UI a slightly more polished and friendly touch. The square borders make it feel just that – square. So I was quite pleased to find Displaperture. Amusingly enough, it even lets you adjust just how far into your screen the border goes.

So, if you miss a bit of curve, check it out:


Filed under: Uncategorized

Introductory Note

I hate starting a blog and then getting stuck on what to say in your very first post. That’s really the reason why I haven’t started one yet. Really, I much prefer to just get right into the swing of things and pretend this blog has always existed. So, I’m going to avoid all that, other than to say, ‘Hey dear visitor, thanks for visiting my blog. Sorry there isn’t much content yet, but there will be soon.’ 😉

– i debug

Filed under: Uncategorized


January 2019
« Jul