« Lets Talk Computers Radio Show | Main | Why Cornerstone Bank Choose Diskeeper To Maintain Their Computer Systems »
August 01, 2007
SATA, NCQ and Disk Fragmentation
A DiskeeperBlog reader recently had this question, and I felt it deserved it's own thread:
"Does NCQ (Native Command Queuing) on the newer SATA drives help in reducing fragmentation, or the impact of fragmentation on HDD performance? I have run across this subject recently on a few sites, and I'm thoroughly confused about what NCQ does to fragmentation." - Durian
---
A very good question and, I agree, a very complicated one. As it turns out, I'm in the midst of a very long white page paper on various data storage technologies and how they relate to fragmentation. In it is a semi-completed "chapter" on SATA, SCSI, FC (Fibre Channel), and SAS (Serial Attached SCSI) disks. I've taken a small excerpt from that section to address your immediate question. I apologize in advance if it reads poorly as it still needs some polish.
I recommend chapter 1 of the "Shortcut Guide to Managing Disk Fragmentation" by Microsoft Product Manager Mike Danseglio, for technical references. Wikipedia can help define some of the technologies described if you need more info.
As seek optimization is interelated with the technology of queuing , I'll describe both and explain the relationship.
An important concept to keep in mind when reading this is that only the file system understands the 1's and 0's as separate files (e.g. Report.Doc). Physical disks operate as block-level devices, unaware of whether an incoming set of data it is asked to read or write is originating from one single file or from a number of different files.
Disk/File defragmentation allows the operating system to only have to pass along a single, or small number, of requests for data to the disk subsystem, rather than a great many.
Seek Optimization :
There are a number of disk-level algorithms to minimize the impact of physical limitations such as rotational latency (waiting for the disk to spin back around). They include variants of elevator-seeking and shortest-seek-first. These algorithms leverage the disk buffer, prioritizing retrieval of data physically closest, by measure of the data's cylindrical location and/or how close the requested data is to the current location of the disk head. These technologies have already been implemented in hard drives by drive manufacturers for many years now. Seek optimization does not eliminate the need to defragment. While it improves on rote data retrieval methods of the past, it cannot and does not account for file fragmentation because it is a "block" based technology. Seek Optimization organizes and prioritizes data retrieval based on physical location of data blocks on the disk platter. It does not operate on a "file object" level. On that note, defragmenters solve "file" fragmentation, and do not get involved in block-level allocation.
NCQ:
Native Command Queuing (NCQ) is a technology that allows a SATA drive to intelligently re-prioritize and queue disk requests while completing others (up to 32). You could say that it is like dynamic multi-tasking at the disk level. SCSI disks have long supported command queuing (TCQ - up to 256) before SATA and NCQ were created. While queuing will improve prioritization strategies and overall seek time for small asynchronous I/O (random/non-linear - in other words, not a large MPEG file), it does not address "file" fragmentation. Command queuing is also "block" based, essentially providing buffer space for seek strategies to be more effective.
In summary, for all the I/O the disk is requested to process at a given time, it cannot optimize its activity for a particular file because it does not know what of that requested I/O is from one file or another. The disk will simply retrieve the blocks of data in the fastest way that they can be. As a example, you may get half of two different files open before you get all of one, and then the other, open.
To answer the original question, NCQ does not solve fragmentation. While the performance benefit that NCQ does provide is irrespective of file fragmentation, they are complimentary.
Posted by Michael at August 1, 2007 01:28 AM
