In my C program i am using very large file(approx 400MB) to read parts of it frequently. But due to large file the performance of the program goes down very badly. It shows very high I/O usage and I/O wait time.
My question is, What are the ways to optimize or tune I/O on linux or how i can get better performance when i have to read a large flat file?
I am using
Red Hat Enterprise Linux ES release 4 (Nahant Update 4)
2.6.9-42.EL, x86_64 GNU/Linux
Did you mmap() the file?
You can upgrade your hard disk to increase the read write capacity.
If it's a local disk you could experiment with hdparm and your filesystems tuning utility (like tune2fs/reisertune/...) to tune it.
No i haven't used mmap() the file. Should i use this?
Is there any specific hard disk type comes which is effective in read/write speed?
I have local disk. Which options u will recommend tuning? I will try few which i understand.
For IDE disks, dma settings are quite important to performance.
$ hdparm -d /dev/hda
/dev/hda:
using_dma = 1 (on)
$ hdparm -I /dev/hda
/dev/hda:
ATA device, with non-removable media
Model Number: Maxtor 6L300R0
Serial Number: L60Z0NBG
Firmware Revision: BAJ41G20
Standards:
Used: ATA/ATAPI-7 T13 1532D revision 0
Supported: 7 6 5 4
Configuration:
Logical max current
cylinders 16383 65535
heads 16 1
sectors/track 63 63
--
CHS current addressable sectors: 4128705
LBA user addressable sectors: 268435455
LBA48 user addressable sectors: 586114704
device size with M = 1024*1024: 286188 MBytes
device size with M = 1000*1000: 300090 MBytes (300 GB)
Capabilities:
LBA, IORDY(can be disabled)
Standby timer values: spec'd by Standard, no device specific minimum
R/W multiple sector transfer: Max = 16 Current = 16
Advanced power management level: unknown setting (0x0000)
Recommended acoustic management value: 192, current value: 0
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5 udma6
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
* SMART feature set
Security Mode feature set
* Power Management feature set
* Write cache
* Look-ahead
* Host Protected Area feature set
* WRITE_VERIFY command
* WRITE_BUFFER command
* READ_BUFFER command
* NOP cmd
* DOWNLOAD_MICROCODE
Advanced Power Management feature set
SET_MAX security extension
Automatic Acoustic Management feature set
* 48-bit Address feature set
* Device Configuration Overlay feature set
* Mandatory FLUSH_CACHE
* FLUSH_CACHE_EXT
* SMART error logging
* SMART self-test
Media Card Pass-Through
* General Purpose Logging feature set
* WRITE_{DMA|MULTIPLE}_FUA_EXT
* URG for READ_STREAM[_DMA]_EXT
* URG for WRITE_STREAM[_DMA]_EXT
* SMART Command Transport (SCT) feature set
* SCT Data Tables (AC5)
Security:
Master password revision code = 65534
supported
not enabled
not locked
frozen
not expired: security count
not supported: enhanced erase
HW reset results:
CBLID- above Vih
Device num = 0 determined by the jumper
Checksum: correct
Those two commands show whether DMA is enabled on an IDE drive, and if it is, what kind. hdparm -d1 /dev/hda will attempt to enable DMA transfer on an IDE drive if it isn't already.
hdparm can also do benchmarks:
$ hdparm -tT /dev/hda
/dev/hda:
Timing cached reads: 292 MB in 2.00 seconds = 145.96 MB/sec
Timing buffered disk reads: 140 MB in 3.04 seconds = 46.08 MB/sec
$
My output is
1.
/dev/hda:
using_dma = 1 (on)
/dev/hda:
ATAPI CD-ROM, with removable media
Model Number: CD-ROM 52x.Max
Serial Number: 1999/04/28
Firmware Revision: 1.12
Standards:
Used: ATAPI for CD-ROMs, SFF-8020i, r2.5
Supported: CD-ROM ATAPI-1
Configuration:
DRQ response: 50us.
Packet size: 12 bytes
Capabilities:
LBA, IORDY(can be disabled)
DMA: sdma0 sdma1 sdma2 mdma0 mdma1 *mdma2
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
3.
/dev/hda:
read() failed: Input/output error
Timing buffered disk reads: read() failed: Input/output error
Do you see anything i can tune?
That's your CDROM. Try hdb or hdc. Your disk may also be SCSI, in which case hdparm won't do much.
well it seems that my disk is sda2
$df- h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 225G 137G 78G 64% /
/dev/sda1 92M 9.4M 78M 11% /boot
none 1004M 0 1004M 0% /dev/shm
Now if i am executing commands of hdparm, the out put are as below.
$hdparm -d /dev/sda2
/dev/sda2:
$hdparm -I /dev/sda2
ATA device, with non-removable media
Model Number: ST3250310AS
Serial Number: 6RY1VV58
Firmware Revision: 3.AAC
Standards:
Supported: 7 6 5 4
Likely used: 7
Configuration:
Logical max current
cylinders 16383 16383
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors: 16514064
LBA user addressable sectors: 268435455
LBA48 user addressable sectors: 488397168
device size with M = 1024*1024: 238475 MBytes
device size with M = 1000*1000: 250059 MBytes (250 GB)
Capabilities:
LBA, IORDY(can be disabled)
Queue depth: 32
Standby timer values: spec'd by Standard, no device specific minimum
R/W multiple sector transfer: Max = 16 Current = 16
Recommended acoustic management value: 208, current value: 0
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
* READ BUFFER cmd
* WRITE BUFFER cmd
* Host Protected Area feature set
* Look-ahead
* Write cache
* Power Management feature set
Security Mode feature set
* SMART feature set
* FLUSH CACHE EXT command
* Mandatory FLUSH CACHE command
* Device Configuration Overlay feature set
* 48-bit Address feature set
SET MAX security extension
* DOWNLOAD MICROCODE cmd
* General Purpose Logging feature set
* SMART self-test
* SMART error logging
Security:
Master password revision code = 65534
supported
not enabled
not locked
frozen
not expired: security count
not supported: enhanced erase
Checksum: correct
$hdparm -tT /dev/sda2
/dev/sda2:
Timing cached reads: 3032 MB in 2.00 seconds = 1514.72 MB/sec
Timing buffered disk reads: 234 MB in 3.02 seconds = 77.47 MB/sec
From the first output it seems that DMA is not enabled. and when i tried to enable it using command below it fails.
$hdparm -d1 /dev/sda2
/dev/sda2:
setting using_dma to 1 (on)
HDIO_SET_DMA failed: Inappropriate ioctl for device
As already explained hdparm is for IDE disks. You can't enable IDE DMA on a disk or driver that's not IDE. It does seem to be able to do benchmarks though, which shows your disk running at rather good speeds indeed.
What really ruins disk performance is random seeking. Every time the head moves the disk has to wait and do nothing until its ready. Could your program be made to process data sequentially instead of seeking all over the place in the file?
Another thing you could do is add more memory. More of the file will be able to stay in cache, which will increase performance a lot.
By adding memory do you mean to add extra RAM disk(hardware) in the system?