Dave's mess

The Middle Name Guesser

I have recently made some improvements to the Middle Name Guesser (one of which was to make it actually work again) and I'd like to take this opportunity to invite you to have it guess your middle name... or your friend's middle name, or your favourite celebrity's middle name.

I have also added a couple of statistics graphs and you can clearly see exactly when I fixed that pesky little bug that only showed up when it actually guessed your middle name correctly. (It was a typo I introduced the last time I edited the file - a strong argument for automated testing if ever I heard one.) At that point it was getting about 1 in 20 guesses correct. Since then it has been steadily improving up to a peak of getting 1 in 4 guesses correct. 1 in 4 guesses correct is better than I ever hoped it would achieve. I was originally thinking that 1 in 10 would be a good result. Now I'm wondering if it will get to 1 in 2...

I expect to see the ratio of correct to incorrect guesses remain relatively unstable until the number of new, unique middle names, first names and last names (the red, blue and purple line) starts flattening out. After that the ratio should only improve as the relationships between the known first and last names and middle names are strengthened.

The air powered car

There's an air powered car that has been causing some hype recently (which, I suppose, is considered "fuel" for this new car. Heh.) and, while it's not all that new, some people are cautiously (and not so cautiously) predicting that "2008 is the year of the air powered car". As a born skeptic, I felt the urge to play devil's advocate.

My first thought was that the compressed air has to come from somewhere and that the process of compressing the air would require energy from more traditional sources. This technology isn't a new way of generating or extracting energy. Much like the talk of Hydrogen-powered cars, this is a new method of storing energy in cars that has been generated somewhere else. Most of these sorts of schemes don't help reduce pollution, they just offset it somewhere else. While this is good for people who live in cities, it's not any better for the planet as a whole.

But there may be more to this plan than just offsetting the pollution. A compressed-air powered car has a few advantages over a Hydrogen powered car: Hydrogen has to be converted from it's pure state into a form with a lower energy content or higher entropy. This is usually achieved by combining it with Oxygen, which is readily found in the atmosphere. The process of combustion usually takes place inside a modified conventional engine or in a Hydrogen based fuel cell, however, both of these methods generate lots of wasted energy. The power extracted from the Hydrogen comes from the expansion of the gases as they combine. The sound and heat energy that is produced at the same time is dissipated into the environment and is wasted.

A compressed-air powered car, on the other hand, can extract the same gaseous expansion based energy as combustion based cars without the loss of the heat and sound-based energy. There has been some discussion (although the results I found were inconclusive) about whether the process of compressing the air was inefficient enough to offset the gains made with the more efficient power stations and in-car decompression process. The end result of reducing waste energy is that not only would the car cause less noise pollution, but the energy used to actually drive the car could be a greater percentage of the total energy available. Less waste is a good thing.

There are, however, a few elements of the article that caused me some concern. The talk of the compressed air driving the pistons which in turn compress the air makes little sense. This is akin to using an electric motor to drive a generator which powers the electric motor. If it worked, it would violate the law of conservation of energy. I suspect (hope) that an over-enthusiastic reporter snuck this into the article rather than quoting directly from a scientist.

The article also makes no mention of the range of the car apart from stating that there is a long-range version that would be fitted with a conventional engine. This suggests to me that this new car would suffer from the same drawbacks that electric cars suffer from: a range so small that the car is limited to the inner-city commute from home to work. After a quick Google and a visit to WikiPedia, it appears that other sites claim the range would be somewhere between 100Km - 200Km. That's great for those who only need that but I won't be swapping the long-range fuel tank in my Pajero for one of these until it comes closer to the same range. Earlier articles regarding the same technology suggest even lower ranges so with the technology getting better and better, hopefully the air car will achieve that goal eventually.

Filling me with confidence again, the rest of the article shows that Negre (The motivation behind the idea) truly understands the problem of wasted energy. Firstly, the direct quote: "The lighter the vehicle, the less it consumes and the less its pollutes and the cheaper it is; it's simple," is very similar to one of the major principles behind low-energy building design. So often, when you design something inefficiently, you find that you need to waste more energy to fix problems with the design. Cars have added weight to deal with the wasted sound and heat energy which, in turn, requires more energy to carry around. Fridges emit all their heat at the back, which often gets trapped and heats the inside of the fridge back up. Fridges have to use extra energy just to remain below room temperature because the air around the fridge is above room temperature. The less wasted energy a car has, the less weight it needs to carry around to deal with the side-effects of the wasted energy. The less weight it has to carry around, the more you can do with the energy you have. In fact, the expansion of a compressed gas will actually draw in heat - the same way a fridge works - meaning the air can then be used for cooling the interior of the car. An air-conditioner and a radiator are two fewer pieces of machinery this car has to carry around thanks to it's more efficient design.

Negre also has plans to use small factories in the same regions where the car is to be sold. This will probably cost slightly more - large scale factories have the advantage of being cheap on a per-car basis - but it will cost the environment less. He stated that the parts would not be shipped to the factory to be assembled but would rather be sourced locally - saving again on the environmental costs of shipping.

It's possible, with the advances in technology we have made, that the whole process may just even turn out cheaper in dollars than shipping the cars half-way around the world. Wasted energy and wasted effort are wasted dollars. If Negre understands this, and I think he does, then this venture should turn a profit for both his bank balance and the environment.

MoneySavingExpert under DDoS attack

Last weekend, MoneySavingExpert (my old employer) was the subject of what appears to be a fairly hefty DDoS attack. It has been reported on several blogs and shortly afterwards on Digg.

There has been much speculation about why it's happening just now and who could be behind it but, as always, without any data to analyse there's no way of making any guess more accurate than a wild stab in the dark. There has also been much wailing and gnashing of teeth about the powerlessness one feels when being attacked by half the internet. Not that the tech team over at Money Saving Towers were wailing or gnashing their teeth, they just got in and fixed the problem. By Sunday afternoon there was a static holding page up which I could actually request and receive in a browser and by Monday morning the site appeared to be back up and running as usual although I think the forums were still down at that time.

There are some things that can be done when you are the victim of a DoS attack. If MoneySavingExpert can survive it, then so can you.

How you deal with a DoS depends greatly on how it's happening. If you don't already know why your site is down, start trying to find the reason. Log files and aggregated statistics are always the first two places I look.

At my current place of employment, we have a series of graphs generated using Orca and RRDTool for each of our servers. These graphs show us everything from CPU load to disk space used to the number of open TCP connections to the machine's uptime. If a particular server is causing the problem then I can load all of its graphs in a single window and scroll down the list looking for anything unusual. If the problem is with a particular website then I can load up just the servers that website affects. If I don't know which part of our system is the cause of the downtime, then I can load them all up.

Unusual patterns in log files can also be an indicator that something is wrong. If I notice that one IP address has requested more web pages than the next ten combined then I start to suspect that something is wrong at that IP address. If I notice that today's log file is twenty times the size of yesterday's log file, then I'm going to want to have a look inside both of them. At this stage, all I'm doing is gathering information because I don't even know if it's a deliberate DoS or just some other sort of site outage. Either way, the logfiles often hold the answer.

There are many different ways a DoS can be caused. Simply flooding a webserver with ten times the normal number of requests it has to deal with is a crude but effective method. This method will often cause your upstream bandwidth provider to start dropping packets because it can't keep up the pace. Even if your webserver could serve all the requests, some of them won't make it all the way there. Other types of DoS exist, however, and it's worth mentioning some of them here.

There are plenty of vulnerabilities in the off-by-one-buffer-overflow category that will cause a program to crash. These are inevitably classed as denial of service vulnerabilities because that's usually all that can be exploited with them. The important thing to note is that you don't need a large botnet or even a small one to cause a DoS to someone using this method. All an attacker needs is a single computer with the ability to anonymise it's payload through something like ToR or a list of proxy servers. Every crash (i.e every request) is going to cause several minutes of downtime.

Another class of DoS attack is caused by requesting a page that causes a lot of resource usage, such as requesting '%' from a badly written search function. If the page is vulnerable, this example will cause the result set of the search to include every row in the database. This will chew up large amounts of CPU and RAM even if it only actually displays the top ten results.

A DoS attacker could also request pages that cause lots of logging to occur, hence filling up the victim's file system. I have actually caused this to happen completely by accident on one guy's website. Apparently, in the space of about half an hour I caused 60GB of log files to be generated on their webserver. Luckily, they knew what I was doing and had my phone number so they could ask me to stop.

These sorts of attacks - the ones that cause resource starvation on your webserver - can be caught with an IDS such as Snort, any decent firewall or a dedicated appliance. Once you can identify the packets that are part of the DoS it is simply a matter of knowing how your firewall/IDS is configured and configuring it to drop those packets.

The other sort of DoS attack - the sort that attacks the services that support your site rather than the site itself - cannot be stopped by you. They will require the people who run the service that failed to do whatever they need to do to survive the attack. In the case of MoneySavingExpert, it appears that they have requested the services of ProLexic, a company that specialises in mitigating the effects of bandwidth-based DDoS attacks. Essentially, ProLexic point all of the victim's traffic at their own servers, filter out the bad requests and pass the remaining requests on to the real webservers. It's a simple but effective tactic that works against the crude but effective attack.

Little Bobby Tables

Ahhh xkcd, you've done it again.

There's not enough security humour in this world.



I want to name my cat Tiddles"><script>alert('Foo!');</script> now, just so that I can put that in as the answer to my secret question on Facebook.

I just remembered that xkcd always put a title tag on every image that contains another little joke. I've replicated the title-tag joke for this comic here as well. If you're using Firefox, you can hover over the image to read it.

So many servers, all hacked.

Yesterday, while trying to track down a problem with one of our forums, I was looking through the validation log and spotted something rather unusual.

The validation log stores all the parameters passed to the forums that failed validation so that we can verify that no legitimate users are being denied access. Parameters include things like which post you are looking at, which thread it's in, which board the thread's in and which page of the thread you are on. Normally, the post number, thread number and page number are all actually numbers but occasionally, somebody thinks it might be a good idea to put something else, like a URL, into the post number parameter.

The result was astounding.
I sat there for minutes, watching the URLs of compromised servers fly past on my screen. In this case, it was a misguided hacking attempt aimed at a completely different piece of software than the one we are running. We didn't have the vulnerability he was trying to exploit. Had it been aimed at the correct software and succeeded, it would have would have changed the parameter so that instead of including a PHP file from the webserver, it would have included a file from someone else's webserver and run that file just as it does when the file is local. The difference is that the code from the other webserver would have installed a rootkit, a command and control interface, a couple of new users and finally sent a message back to it's owner telling him where we were.

Unfortunately, people who try to seize control of other people's webservers are a paranoid lot. They don't usually just start hacking from their home computer and head straight for the target. They will use Tor or an anonymous proxy to mask their true identities. They'll use webservers that they have already cracked to help crack new webservers. In this case, tracing the hacking attempt back to where it came from only lead us to another compromised server with a web-based command and control page and the file required to hack other servers.

I didn't pursue it any further for several reasons: I'm not paid to hunt down crackers, it would have been illegal for me to use the compromised server to find out where it had been compromised from and it was an unsuccessful attempt to exploit a vulnerability we didn't even have. Out of interest, I did quickly grep through the entire set of validation logs just to see how many of these attempts there were and from how many already-compromised webservers. The result was astounding. I sat there for minutes, watching the URLs of compromised servers fly past on my screen.

I wasn't all that surprised to see lots of hacking attempts. Just put a machine on the internet running Snort for a day and you'll understand why. What did surprise me was the sheer number of already compromised servers sitting out there. Do people not have intrusion detection systems ? Do they not check their log files ? Has somebody like me not already noticed that their server has been hacked and emailed to let them know ? (For the record, I did email the admin of the first server but once I found the hundreds or thousands in the log files I decided that it was a bit much effort for me...)

Does security not matter to these people ?

I suspect that's the answer. Most people are on the net to create something. They aren't interested in learning all about computer security and how to secure their machines. They just want to create their own little corner of the web where they can do as they please.