Disabling Command Completion Coalescing (CCC) in SATA AHCI - linux

Question

I'm working on 2.6.35.9 version of the Linux kernel and am trying to disable Command Completion Coalescing.
The output of lspci is as shown below:
00:00.0 Host bridge: Intel Corporation 82P965/G965 Memory Controller Hub (rev 02)
00:01.0 PCI bridge: Intel Corporation 82P965/G965 PCI Express Root Port (rev 02)
00:19.0 Ethernet controller: Intel Corporation 82566DC Gigabit Network Connection (rev 02)
00:1a.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #4 (rev 02)
00:1a.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #5 (rev 02)
00:1a.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI Controller #2 (rev 02)
00:1c.0 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 1 (rev 02)
00:1c.4 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 5 (rev 02)
00:1d.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #1 (rev 02)
00:1d.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #2 (rev 02)
00:1d.2 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #3 (rev 02)
00:1d.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI Controller #1 (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev f2)
00:1f.0 ISA bridge: Intel Corporation 82801HH (ICH8DH) LPC Interface Controller (rev 02)
00:1f.2 RAID bus controller: Intel Corporation 82801 SATA RAID Controller (rev 02)
00:1f.3 SMBus: Intel Corporation 82801H (ICH8 Family) SMBus Controller (rev 02)
01:00.0 VGA compatible controller: nVidia Corporation G72 [GeForce 7300 LE] (rev a1)
04:03.0 Mass storage controller: Promise Technology, Inc. PDC20268 (Ultra100 TX2) (rev 02)
I have Native Command Queuing enabled on my drives.
I was looking at the Serial ATA AHCI 1.3 Specification and found on page 115 that -
The CCC feature is only in use when CCC_CTL.EN is set to ‘1’. If CCC_CTL.EN is set to ‘0’, no CCC
interrupts shall be generated.
Next, I had a look at the relevant code (namely, the files concerning AHCI) for this version of the kernel but wasn't able to make any progress. I found the following macro enum HOST_CAP_CCC = (1 << 7) in drivers/ata/ahci.h, but I'm not sure how this should be modified to disable command coalescing.
Can someone please assist me in identifying how CCC can be disabled? Thank you!
In response to gby's comment:
I conducted an experiment where I issued requests of size 64KB from my driver code. 64KB corresponds to 128 sectors (each sector = 512 bytes).
When I look at the response timestamp differences, here is what I find:
Timestamp | Timestamp | Difference
at | at | in microsecs
Sector 255 - Sector 127 = 510
Sector 383 - Sector 255 = 3068
Sector 511 - Sector 383 = 22
Sector 639 - Sector 511 = 22
Sector 767 - Sector 639 = 12
Sector 895 - Sector 767 = 19
Sector 1023 - Sector 895 = 13
Sector 1151 - Sector 1023 = 402
As you can see, the response timestamp differences seem to suggest that the write completion interrupts are being batched into one and then one single interrupt is being raised, which might explain the really low numbers in tens of microseconds.
Also, when conducting this experiment, the on-disk write cache was disabled using hdparm.
Clearly, there is some interrupt batching involved here which I need to disable so that an interrupt is raised for each and every write request.
UPDATE:
Here is another experiment that I tried.
Create a bio structure in my driver and call the __make_request() function of the lower level driver. Only one 2560 bytes write request is sent from my driver.
Once this write is serviced, an interrupt is generated which is intercepted by do_IRQ(). Finally, the function blk_complete_request() is called. Keep in mind that we are still in the top half of the interrupt handler (i.e., interrupt context, not kernel context). Now, we compose another struct bio in blk_complete_request() and call the __make_request() function of the lower level driver. We record a timestamp at this point (say T_0). When the request completion callback is obtained, we record another timestamp (call it T_1). The difference - T_1 - T_0 - is always above 1 millisec. This experiment was repeated numerous times, and each time, the destination sector affected this difference - T_1 - T_0. It was observed that if the destination sectors are separated by approximately 350 sectors, the time difference is about 1.2 millisec for requests of size 2560 bytes.
Every time, the next write request is sent only when the previous request has been serviced. So, all these requests are chained and the disk has to service only one request at a time.
My understanding is that since the destination sectors of consecutive requests have been separated by a fairly large amount, by the time the next request is issued, the requested sector would be almost below the disk head and thus the write should happen immediately and T_1 - T_0 should be small (at least < 1 millisec).
The Serial ATA AHCI 1.3 Specification (page 114) states that:
When a software specified number of commands have completed or a software specified
timeout has expired, an interrupt is generated by hardware to allow software to process completed commands.
My guess is that this timer maybe the reason why the latency of each request is above 1 millisec. That's why I need to disable CCC.
I did mail the author - Jeff Garzik - but I haven't heard from him yet. Is he a registered user on stackoverflow? If yes, I could PM him...
The HDD we are using is: WD Caviar Black (Model number - WD1001FALS).
Anyone? :-(

Answer 1

AFAIK, HBA capabilities bit7(CCC supported) is RO and you can check it first to see if CCC supported. Then by spec you can disable CCC by setting CCC_CTL.EN because it is RW
Do you try to clear it then conduct your experiment ?