Sar -u generates multiple column headers in csv file

Shell Programming and Scripting
1

Hi All,

The below sar -u command generates multiple column headers in csv file

Expected output should print column headers only once in the csv file

shell script:

$cat sar_cpu_EBS.sh

#!/bin/bash


while [ 1 -eq 1 ]; do
sar -u 15 1 | awk '/^[0-9]/ {print $1,$2,$4,$6,$7}' | tr -s ' ' ',' >> /sar_cpu_EBS_`date +%m%d%y`.csv
sleep 5
done


Output the shell script provides:

$cat sar_cpu_EBS.csv

10:23:03,AM,%user,%system,%iowait
10:23:18,AM,0.69,0.60,0.05
10:23:23,AM,%user,%system,%iowait
10:23:38,AM,0.74,0.68,0.10
10:23:43,AM,%user,%system,%iowait
10:23:58,AM,0.68,0.54,0.05
10:24:03,AM,%user,%system,%iowait
10:24:18,AM,0.45,0.56,0.03
10:24:23,AM,%user,%system,%iowait
10:24:38,AM,0.46,0.49,0.06
10:24:43,AM,%user,%system,%iowait
10:24:58,AM,0.52,0.60,0.33
10:25:03,AM,%user,%system,%iowait
10:25:18,AM,0.42,0.50,0.02
10:25:23,AM,%user,%system,%iowait
10:25:38,AM,0.48,0.49,0.03
10:25:43,AM,%user,%system,%iowait
10:25:58,AM,0.41,0.45,0.03
10:26:03,AM,%user,%system,%iowait
10:26:18,AM,0.47,0.58,0.04
10:26:23,AM,%user,%system,%iowait
10:26:38,AM,0.60,0.59,0.08
$


Expected output should be:

$cat sar_cpu_EBS.csv

10:23:03,AM,%user,%system,%iowait
10:23:18,AM,0.69,0.60,0.05
10:23:38,AM,0.74,0.68,0.10
10:23:58,AM,0.68,0.54,0.05
10:24:18,AM,0.45,0.56,0.03
10:24:38,AM,0.46,0.49,0.06
10:24:58,AM,0.52,0.60,0.33
$

Could anyone please let me know how to achieve this!

Thanks for your time!

Regards,
a1_win

2

Try this... not tested...

#!/bin/bash

hdr=1
while true; do
  sar -u 15 1 |  awk '/user/{if(hdr){print $1,$2,$4,$6,$7};next} /^[0-9]/{print $1,$2,$4,$6,$7}' OFS=, hdr=$hdr >> /sar_cpu_EBS_`date +%m%d%y`.csv
  sleep 5
  hdr=0
done

--ahamed

3

Thanks Ahamed! This is working as per the expected output!

Please let me know what is meant by "OFS=," in the shell script you updated as above

Thanks for your time!

Regards,
a1_win

4

OFS is the Output Field Separator. awk's default field separator is space which is why you were using tr to translate it to comma. So, here we define the OFS to be comma.
Try the statement with different and you will understand.

--ahamed

5

Hi Ahamed,

Thanks but when I try using the same logic in the shell script to find out the page statistics, the headers are being repeated

-bash-3.2$ sar -B 5 1
Linux 2.6.32-300.41.1.el5uek   04/09/2013

02:13:08 PM  pgpgin/s pgpgout/s   fault/s  majflt/s
02:13:13 PM      0.00      2.41   1674.10      0.00
Average:         0.00      2.41   1674.10      0.00
-bash-3.2$

shell script:

 cat sar_page_EBS.sh

#!/bin/bash

hdr=1
while true; do
sar -B 5 1 | awk '/pgpgin\/s/{if(hdr){print $1,$2,$3,$4};next} /^[0-9]/{print $1,$2,$3,$4}'  OFS=, hdr=$hdr >> /sar_page_EBS_`date +%m%d%y`.csv
sleep 1
done

The output provided by the script:

-bash-3.2$ cat sar_page_EBS.csv

02:05:36,PM,pgpgin/s,pgpgout/s
02:05:41,PM,0.00,0.00
02:05:42,PM,pgpgin/s,pgpgout/s
02:05:47,PM,0.00,2.38
02:05:48,PM,pgpgin/s,pgpgout/s
02:05:53,PM,0.00,6.37
02:05:54,PM,pgpgin/s,pgpgout/s
02:05:59,PM,0.00,0.00
02:06:00,PM,pgpgin/s,pgpgout/s
02:06:05,PM,0.00,3.98
02:06:06,PM,pgpgin/s,pgpgout/s
02:06:11,PM,0.00,11.22
02:06:12,PM,pgpgin/s,pgpgout/s
02:06:17,PM,0.00,3.96
02:06:18,PM,pgpgin/s,pgpgout/s
02:06:23,PM,0.00,43.91
02:06:24,PM,pgpgin/s,pgpgout/s
02:06:29,PM,0.00,9.54
02:06:30,PM,pgpgin/s,pgpgout/s
02:06:35,PM,0.00,0.00
02:06:36,PM,pgpgin/s,pgpgout/s
02:06:41,PM,0.00,13.60
02:06:42,PM,pgpgin/s,pgpgout/s
02:06:47,PM,0.00,0.00

Expected output what should be(The columns should be printed once):


-bash-3.2$ cat sar_page_EBS.csv

02:05:36,PM,pgpgin/s,pgpgout/s
02:05:41,PM,0.00,0.00
02:05:47,PM,0.00,2.38
02:05:53,PM,0.00,6.37
02:05:59,PM,0.00,0.00
02:06:05,PM,0.00,3.98
02:06:11,PM,0.00,11.22
02:06:17,PM,0.00,3.96
02:06:23,PM,0.00,43.91
02:06:29,PM,0.00,9.54
02:06:35,PM,0.00,0.00
02:06:41,PM,0.00,13.60
02:06:47,PM,0.00,0.00

Please let me know what is missing here?

Thanks for your time!

Regards,
a1_win

6

You missed hdr=0 inside the while loop.
Check post#2

--ahamed

7

Yes Ahamed! I figured it out and wanted to update the post!!

Thanks again Ahamed! This worked for me and I get the desrired output!!

Regards,
a1_win