Defrag (pt2) You can compare products - but not easily.
It seems to be some kind of universal rule that the moment I describe a post here as "Part One" intending it to start a series, something happens to drag my attention away or make the issue irrelevant *sigh*. Still after a long time waiting, this is part 2 of my comments and thoughts about disk fragmentation and the defraggers that just love to tame it.
In part one, I talked about how you can't measure how well defraggers work against one another very easily. Claus Valca wrote a nice article about defraggers over on Grand Stream Dreams where he talks about ways the process can still be valuable and asks about ways of evaluating the different products. I can think of one way of comparing different products, but while it's honest and fair in terms of what it tests, it probably doesn't answer all the questions you should be asking on the subject. Still it's a start right?
Just to make things clear, I'm using the definitions of File System Fragmentation and Defragmentation found on Wikipedia. I'm not claiming those definitions are perfect but they're at least as good as any I could write myself and they already exist.
The Test
Pick a 'neutral' program to measure the level of fragmentation. For the purposes of this test, the results of analysing the disk with this program will be the only results that are used to compare one product to another.
Decide if you want to do a file-system 'performance' test. If you do, I suggest copying lots of files of varying sizes onto the disk or partition you are using for your tests for this part of the test regime. Try to make sure that the source medium that you grab the files from is fast (because you don't want it to be the slowest point of the test, because you're not testing the speed of the source) and performs consistently (because you want a fair test, right?). Yes, this will be quite difficult to achieve.
Some drive imaging programs allow you to take a "raw" backup image of a disk partition or even of an entire hard disk. I'm linking to Acronis True Image's FAQ here as an example of a product that I'm comfortable with and which offers the facility, though there are other choices out there.
Using such an imaging system, you can create a backup of a test computer's hard disk after you've set it up in such a way as to leave it fragmented enough for your tests.
If you're performing this test on Windows Vista, disable the built-in defragmentation routines NOW, before you do anything else.
For such a test I'd suggest using a relatively small hard disk and creating fragmentation by installing lots of programs and copying large files (got a couple of Linux ISOs doing nothing in your archives?) around the disk, then un-installing some of the programs and deleting some of the images. Considering that NTFS does try to avoid disk fragmentation in the first place you may actually find this bit to be the most difficult part ;-)
Once this is done, ensure that things are set up how you want them, and then disable any 'automatic update' routines for the OS, applications, antivirus, etc. This is very important because you want to ensure as much as possible that what you're measuring in these tests are changes made by the defragger and not by anything else; it's hardly a fair test if windows update downloads and installs an extra 100Mb of patches halfway through one of your test rounds, or if the virus scanner decides that now would be a great time to open up every single zip, rar and jar archive on your disk and do a deep scan of their contents.
If the partition / disk you're testing does not include the operating system you're running the tests from, you will need to 'clean' the OS between each test to remove one defragger before you test the next. The only reliable way to do this would be with taking an image of the operating system before the start of the tests and replacing the OS between each test round.
I suggest two rounds for each product - A simple dumb "file defragmentation" pass where the simple basics are compared, a "like for like" comparison of the basic ability of each product to shuffle a pack of files on your hard disk - this round will probably be rather boring as you'd expect competent products to all perform to a high standard with the simple stuff. I'd then suggest another pass where each product gets to strut its stuff with all the fancy options turned on.
What are you testing? Ensure you understand the products you are testing. If a particular product works in a substantially different way from others (AShampoo's Magic Defrag and Vista's own built-in defrag spring to mind here) then how can you compare it to other products in a meaningful way? If you test a product by a set of rules that don't suit it then you've tested your own ability to evaluate these products and failed badly.
That seems like a lot of work. Is it worth it?
My personal feelings are that for most people, all that stuff above is far more effort than it's worth. Without all that and probably a few other things too, you're not doing a fair comparison of different defrag tools. To be honest, unless you've got specialist needs then you probably don't need to defragment your disks too often these days, as Claus says, it's still important but no longer vital, as it used to be.
Given my previous poor record for following up posts in a timely manner I may be tempting fate saying this, but in a 3rd part to this series I'm going to try and address a few final points, in particular specialist defragmentation requirements.