This isn't a new article. Actually, it is quite old. It should also be pointed out that the Technet article author states "And while it's good to have free space, it's not good if it's fragmented."

Diskeeper is a study in contradictions and inconsistencies when it comes to free space fragmentation/consolidation. For years, you folks have claimed that free space consolidation doesn't do anything to improve performance - in spite of your own documentation stating otherwise.

Diskeeper also has (with V10) a Comprehensive Defragmentation method that performs additional free space consolidation. If free space consolidation isn't necessary and wastes system resources (your words), why do you even have this defragmentation method? I've tried this method and found it completely useless. New files were created fragmented faster than Diskeeper was able to consolidate the free space on the drive. I'm not talking about leaving a few free space chunks behind. Diskeeper would leave thousands of free space chunks behind.

Keep in mind that I was a Diskeeper customer for years - until recently. I'm a firm believer in defragmentation but was always puzzled why after defragmenting with Diskeeper, the disk performance just wasn't where I thought it should be. I discovered that while Diskeeper was defragmenting files, it was neglecting free space consolidation. I've since discovered that free space consolidation DOES matter and it does improve drive performance. Granted, Diskeeper is quick to defragment files in most instances. However, it just simply doesn't provide the drive performance that I've been able to get by defragmenting files AND consolidating free space.

How convenient of you to leave out the rest of the paragraph in the technet article. Here is the subject on Free Space Fragmentation:
---------
Free Space Fragmentation
A partially full disk contains unused space, known as free space. Ideally, this space would be available in a few contiguous portions of the disk. And while it's good to have free space, it's not good if it's fragmented. Free space fragmentation refers to file space that's broken into small pieces, rather than joined together. This type of fragmentation results in slowed performance because of the time it takes for the disk head to move to different points on the disk to find free space and then write the file. Fragmented free space also increases the possibility of file fragmentation; when a file is larger than the space it's being written to, the file fragments.-------
I have the feeling that this post will not appear.

Somone can get another person to agree with them by taking a quote out of context. When you read the rest of the MS article, they recommend having the free space all together. Here is the rest of the quote:
And while it's good to have free space, it's not good if it's fragmented. Free space fragmentation refers to file space that's broken into small pieces, rather than joined together. This type of fragmentation results in slowed performance because of the time it takes for the disk head to move to different points on the disk to find free space and then write the file. Fragmented free space also increases the possibility of file fragmentation; when a file is larger than the space it's being written to, the file fragments.

Guys, I'm new to Diskeeper (v10, but not defragmentation in general) so I don't know Diskeeper's history, but I understood Paul's comments to mean that the extra effort of consolidating a few blocks of free space into one does not provide a significant benefit. I also believe that the MS article supports this notion, they just didn't choose to distinguish between very-fragmented, minimally-fragmented, and totally-unfragmented free space.

Sure, having one large consolidated free space is better than the free space being distributed across the volume in thousands of chunks, but I doubt that one large free space is significantly better than a handful. This could be seen as the free space equivalent of the "Efficiently defragment large files" option (in v10, I don't know about earlier versions).

As a small [contrived] example, imagine a 700GB volume currently has 400GB used space and 300GB free space split in alternating 100GB blocks, ie. "X.X.X.X" where "X" is 100GB/used and "-" is 100GB/free. Do you really think that there would be substantial performance benefit in moving 200GB worth of data in order to have a single 300GB block of free space ("XX...XX") than leaving the drive in it's current state and handing any fragmented files as they occur (and they will occur in either state anyway)?

Hi Steven,

You hit the nail on the head. And while the previously mentioned MS article did not elaborate, any ambiguity is succinctly abated in the comment below:

http://www.microsoft.com/windows2000/en/advanced/help/default.asp?url=/windows2000/en/advanced/help/defrag_faq_free_space.htm

But Diskeeper will leave a small file in the middle of a large block of free space as long as it isn't fragmented. Which means that I have to find another program to do free space consolidation. I'm not really happy with that. I can see how moving larger chunks to consolidate small areas of free space is counter-productive but moving the little fragments in the middle of large areas of free space would prevent a lot of future fragmentation.
You could make a rather simple algorithm based on the ratio of the size of the free space on either side to the size of the used chunk with a upper limit on the chunk size.

Hi Morten,

One file in the middle of a large block of free space could only, worst case, fragment a really large file into two pieces. Performance wise, that is negligible, but from an effort perspective, it may possibly mean Diskeeper has to do more later, rather than less sooner. Obviously, left unhandled this will get worse over time - as more files fragment. The file allocation algorithm in Windows 2000 and newer (NTFS) is more complicated. We've documented this in an audio-video slide in the Diskeeper Multimedia Tour (in case you are interested). http://www.diskeeper.com/diskeeper/tour/index.html

Per Microsoft, 64mb blocks are the size after which defragmentation benefit is less significant. That is why the Vista defrag does take as much effort to fully defrag larger files. That's there opinion, which we agree with - up to a certain point.

I do get your point, and can't say for sure (I'm not that familiar with the algorithm) the point at which we will or won't move that "hanging" file(s). It's something for us to look into - thanks.

If I want to create a 2Gbyte file, say for VMware, then it would be nice to have it in one place. Why not simply provide an option to clear such a space?

Hi Jon,

Given we need to do a lot of development testing, we use virtualization technology here at Diskeeper.

I absolutely agree that you don't want performance degradation on vmdk files (VMware's virtual disk format). Having a vmdk file in a few pieces won't degrade performance. If that file is in a large numner of pieces it certainly will be a problem.

That said, you most likely have 2GB of free space in a single chunk already, if not, and the file is fragmented when written, Diskeeper will quickly and automatically detect the fragmentation and fix it. The idea here is that Diskeeper takes the burden away from you having to run some manual operation prior to setting up a VM.

Another point to keep in mind is that just because you have a large chuck of free space, there is no guarantee the operating system (OS) will choose that spot. It may well try to create that large file in some smaller free space segment. Where the OS write files is up to the OS.

The best situation, especially if you use dynamically expanding VMs (e.g. 2GB that can grow to 4GB) it is best to keep these files on separate volumes away from other files. Ideally you add another spindle into your existing PC and dedicate that new disk to the VM.

Whe it saays diskeeper is free/.Why is it then NOT free.

Hi James,

Could you please explain where the statement is coming (it says) from and exactly what it is that is NOT free?

I have about 500GB of hard-disk space on my Vista machine of which only 30GB is occupied. I want to create partitions in that 470 GB of free space, however, when I tried, Vista would not let me shrink the current 500GB volume by more than 100GB. Why, becuase there is some damn file sitting at the 400GB boundary. I turned off snapshots and paging file to no avail. Vista defragmentation won't move that file closer to the beginning of the HD.

So now I need to shell out money on a software that would do free-space defragmentation OR partitioning software that can move files to make partition.

Hi Atiq,

Remember that you can use Diskeeper free for 30 days. If you are only interested in solving this problem one time - the trial should suffice.

Free Space must exist on the drive so files can be moved around and defragmented.

Is not relevant that a single chunk of free space exist, (actually several chunks seem better) but contiguous blocks of
free space are required to defrag big (or not so big) files, right?

It seems clear to me that defraggers should try to consolidate free space as a first step, so whenever a big file move is
required, such space is available. Am I wrong ?

If all contiguous free space is used just to consolidate Files and Folders (sorted by name or date) the operation can take

ages under an heavy fragmentation / low free space (or not so low) environment.

Also take in account that in NTFS 12,5% of the volume is reserved for MFT expansion purposes (although shown as free, and

usable after "other" space exhaustion), and cannot be used for defrag operations.

Hi DK Eval,

Yes, you do have to have free space - correct. To defrag a larger fragmented file, yes, consolidating the space beforehand is a likely approach. Diskeeper employs, thoughout it's real time operations, a regular evaluation of what "strategy" to use - files then free space or vice versa - depending on what is sees on the volume. This is what we refer to as "smart defrag".

The 12.5% reserved zone is now usable in modern Windows OS'es for defrag. That said, it's not a good idea to leave a file there. That reserved zone is also fairly dynamic and can shrink, as the MFT fills up, returning the space back for general storage.