Thursday, 10 September 2009

Disk performance analyzing in Linux

Disk performance analyzing in Linux


  1. Abstract:

I need to check disk activity during a day to analyze disk load in peak time. Because I prefer to show to my colleagues a graphical data, I have to collect it to plain text file and prepare graphics later.

  1. Collecting data

I choose iostat tool as it collects many disks data.

#/usr/bin/iostat -d -x

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sda 0.03 4.59 0.11 2.18 3.66 54.53 25.36 0.34 147.20 9.54 2.19
sda1 0.03 4.59 0.11 2.18 3.66 54.53 25.36 0.34 147.20 9.54 2.19
sda2 0.00 0.00 0.00 0.00 0.00 0.00 22.13 0.00 12.07 6.07 0.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 15.42 0.00 4.97 2.67 0.00
sdb1 0.00 0.00 0.00 0.00 0.00 0.00 15.16 0.00 4.86 2.37 0.00

Unfortunately "The first report generated by the iostat command provides statistics concerning the time since the system was booted. Each subsequent report covers the time since the previous report. All statistics are reported each time the iostat command is run. " - extract from iostat man page.

It means that I can't start it from script and redirect output elsewhere. But Linux has tee utility which read from standard input and write to standard output and files. Also I need time stamp for each portion of performance data. I use the following command:

$ /usr/bin/iostat -d -x -t 10|tee file
Linux 2.6.28-15-generic (rifle-desktop) 24/09/09 _i686_ (4 CPU)

24/07/09 23:18:14
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sda 1.88 19.00 7.88 4.58 570.58 188.97 60.98 0.80 64.31 2.04 2.54
sda1 0.01 0.08 0.00 0.00 0.06 0.65 146.09 0.00 20.29 19.40 0.01
sda2 0.00 0.00 0.00 0.00 0.00 0.00 2.00 0.00 68.00 68.00 0.00
sda5 1.87 18.92 7.88 4.57 570.51 188.32 60.95 0.80 64.33 2.04 2.53
sdb 0.06 5.73 0.10 0.06 16.62 46.28 395.72 0.01 62.91 4.78 0.08
sdb1 0.06 5.73 0.10 0.06 16.62 46.28 396.39 0.01 63.01 4.78 0.08

24/07/09 23:18:24
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sda 0.00 9.20 2.00 5.20 64.00 115.20 24.89 0.03 4.28 2.00 1.44
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda5 0.00 9.20 2.00 5.20 64.00 115.20 24.89 0.03 4.28 2.00 1.44
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sdb1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

  1. Prepare for analysis

Now I have the file with many repeating blocks like this:

Time: 11:28:04 AM

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util

sda 0.03 4.59 0.11 2.18 3.66 54.55 25.36 0.34 147.24 9.54 2.19

sda1 0.03 4.59 0.11 2.18 3.66 54.55 25.36 0.34 147.24 9.54 2.19

sda2 0.00 0.00 0.00 0.00 0.00 0.00 22.13 0.00 12.07 6.07 0.00

sdb 0.00 0.00 0.00 0.00 0.00 0.00 15.42 0.00 4.97 2.67 0.00

sdb1 0.00 0.00 0.00 0.00 0.00 0.00 15.16 0.00 4.86 2.37 0.00


Time: 11:28:14 AM

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util

sda 0.00 1.20 0.00 0.80 0.00 15.98 20.00 0.02 25.62 14.50 1.16

sda1 0.00 1.20 0.00 0.80 0.00 15.98 20.00 0.02 25.62 14.50 1.16

sda2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

sdb1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

I have to remove empty lines and repeating header and put timestamp in the beginning of each line with data.


    1. Remove empty lines and line with Device text on it

#sed -e "/^$/d;/Device/d" file>ready.txt

    1. Insert timestamp from Time line in first position of each line with data and omit line with "Time:"

File with awk instructions

#cat test

$1 == "Time:" {

tm = $2

tm1 = $3

next

}

$1 != "Time" {

print tm, tm1, $0

}

Make changes by executing:

#awk -f test ready.txt >ready.csv

Now I have something like this

#head -n 5 ready.csv

03:42:08 PM sda 0.01 4.24 0.04 2.10 0.49 25.52 24.29 0.31 144.10 9.81 2.10

03:42:08 PM sda1 0.01 4.24 0.04 2.10 0.49 25.52 24.29 0.31 144.10 9.81 2.10

03:42:08 PM sda2 0.00 0.00 0.00 0.00 0.00 0.00 22.13 0.00 12.07 6.07 0.00

03:42:08 PM sdb 0.00 0.00 0.00 0.00 0.00 0.00 15.46 0.00 4.92 2.61 0.00

03:42:08 PM sdb1 0.00 0.00 0.00 0.00 0.00 0.00 15.20 0.00 4.81 2.30 0.00

This file can be easily analyzed.

No comments: