Formatting a file using shell script

Shell Programming and Scripting
1

I have a file with contents something similar to as below :

SVCNAME,12m-1a,1a-2a,2a-3a,3a-4a,5a-6a,6a-7a,7a-8a,8a-9a,9a-10a,10a-11a,11a-12n,12n-1p,13p-14p,14p-15p,15p-16p,16p-17p,17p-18p,18p-19p,19p-20p,20p-21p,21p-22p,22p-23p,11p-12m, TOTALS
svc100,145/0.04,124/0.04,72/0.04,2/0.14,2/0.18,67/0.04,299/0.04,392/0.04,419/0.04,497/0.04,418/0.04,705/0.03,831/0.02,118/0.04,149/0.04,109/0.04,113/0.04,44/0.04,56/0.04,74/0.04,16/0.04,23/0.04,112/0.04,       4787/0.03
svc200,159/0.07,166/0.09,89/0.10,6/0.06,0/0.00,33/0.11,250/0.12,361/0.08,344/0.09,592/0.07,480/0.07,422/0.08,222/0.09,126/0.07,118/0.07,148/0.07,202/0.05,86/0.05,88/0.05,76/0.05,18/0.06,15/0.08,121/0.06,  4122/0.08
svc300,157/0.04,154/0.04,78/16622.60,4/0.03,4/0.06,41/0.04,197/0.06,328/0.04,317/0.04,539/0.04,478/0.04,402/0.04,227/0.04,148/0.04,156/0.04,185/0.04,242/0.03,111/0.03,120/0.03,82/0.03,17/0.04,17/0.03,96/0.03,  4100/316.27

The first row shows time
The last column is the avg.
other columns can be read as : no. of times svc was called/avg response time
The o/p should show the columns matching to or greater than the last coulmn's Avg response time (i.e. Avg)

2

You can do something like that :

nawk -F '[,/]' '
   NR==1 {
      for (c=2; c<NF; c++) Times[c-1] = $c;
      TimesCount = NF-2;
      next;
   }
   {
      avg_total = $NF;
      svcname   = $1;
      out       = "";
      field     = 3;
      for (c=1; c<TimesCount; c++) {
          avg = $field;
          if (avg > avg_total) out = out " "  Times[c] " (" avg ")";
          field += 2;
      }
      if (out) print svcname " (" avg_total ") : " out;
   }
' inputfile

Inputfile :

SVCNAME,12m-1a,1a-2a,2a-3a,3a-4a,5a-6a,6a-7a,7a-8a,8a-9a,9a-10a,10a-11a,11a-12n,12n-1p,13p-14p,14p-15p,15p-16p,16p-17p,17p-18p,18p-19p,19p-20p,20p-21p,21p-22p,22p-23p,11p-12m, TOTALS
svc100,145/0.04,124/0.04,72/0.04,2/0.14,2/0.18,67/0.04,299/0.04,392/0.04,419/0.04,497/0.04,418/0.04,705/0.03,831/0.02,118/0.04,149/0.04,109/0.04,113/0.04,44/0.04,56/0.04,74/0.04,16/0.04,23/0.04,112/0.04,       4787/0.03
svc200,159/0.07,166/0.09,89/0.10,6/0.06,0/0.00,33/0.11,250/0.12,361/0.08,344/0.09,592/0.07,480/0.07,422/0.08,222/0.09,126/0.07,118/0.07,148/0.07,202/0.05,86/0.05,88/0.05,76/0.05,18/0.06,15/0.08,121/0.06,  4122/0.08
svc300,157/0.04,154/0.04,78/16622.60,4/0.03,4/0.06,41/0.04,197/0.06,328/0.04,317/0.04,539/0.04,478/0.04,402/0.04,227/0.04,148/0.04,156/0.04,185/0.04,242/0.03,111/0.03,120/0.03,82/0.03,17/0.04,17/0.03,96/0.03,  4100/316.27

Jean-Pierre.
Output :

svc100 (0.03) :  12m-1a (0.04) 1a-2a (0.04) 2a-3a (0.04) 3a-4a (0.14) 5a-6a (0.18) 6a-7a (0.04) 7a-8a (0.04) 8a-9a (0.04) 9a-10a (0.04) 10a-11a (0.04) 11a-12n (0.04) 14p-15p (0.04) 15p-16p (0.04) 16p-17p (0.04) 17p-18p (0.04) 18p-19p (0.04) 19p-20p (0.04) 20p-21p (0.04) 21p-22p (0.04) 22p-23p (0.04)
svc200 (0.08) :  1a-2a (0.09) 2a-3a (0.10) 6a-7a (0.11) 7a-8a (0.12) 9a-10a (0.09) 13p-14p (0.09)
svc300 (316.27) :  2a-3a (16622.60)
3

Thanks bud..

---------- Post updated at 06:03 AM ---------- Previous update was at 05:02 AM ----------

one more requirement has arose :
The threshold is 0.1 for Avg response time. So, only svcs with avr respose time greater than or equal to 0.1 should be displayed.

So in this case, the o/p should be :

svcname 2a-3a Avg Rep
svc300 78/16622.60 4100/316.27

4

This new version of script print count and average :

nawk -F '[,/]' '
   NR==1 {
      for (c=2; c<NF; c++) Times[c-1] = $c;
      TimesCount = NF-2;
      next;
   }
   {
      cnt_total = $(NF-1)+0;
      avg_total = $NF+0;
      svcname   = $1;
      out       = "";
      field     = 3;
      for (c=1; c<TimesCount; c++) {
          cnt = $(field-1);
          avg = $field;
          if (avg > avg_total) out = sprintf("%s %s (%s/%s)", out, Times[c], cnt, avg);
          field += 2;
      }
      if (out) printf "%s (%s/%s) : %s\n",  svcname, cnt_total, avg_total, out;
   }
'inputfile

Output :

svc100 (4787/0.03) :  12m-1a (145/0.04) 1a-2a (124/0.04) 2a-3a (72/0.04) 3a-4a (2/0.14) 5a-6a (2/0.18) 6a-7a (67/0.04) 7a-8a (299/0.04) 8a-9a (392/0.04) 9a-10a (419/0.04) 10a-11a (497/0.04) 11a-12n (418/0.04) 14p-15p (118/0.04) 15p-16p (149/0.04) 16p-17p (109/0.04) 17p-18p (113/0.04) 18p-19p (44/0.04) 19p-20p (56/0.04) 20p-21p (74/0.04) 21p-22p (16/0.04) 22p-23p (23/0.04)
svc200 (4122/0.08) :  1a-2a (166/0.09) 2a-3a (89/0.10) 6a-7a (33/0.11) 7a-8a (250/0.12) 9a-10a (344/0.09) 13p-14p (222/0.09)
svc300 (4100/316.27) :  2a-3a (78/16622.60)

Please show us the required output for your sample datas.

Jean-Pierre.

5

The o/p I am looking for should be :

svcname 2a-3a Avg Rep
svc300 78/16622.60 4100/316.27

Here, since only svc300 is having avg reponse time >= 0.1 (i.e. 316.27) it should display at what time we saw the resonse time >0.1 (which is this case is 2a-3a i.e. 16622.60)

6

The output format is not fixed :

nawk -F '[,/]'  -v Threshold=0.1 '
   NR==1 {
      for (c=2; c<NF; c++) Times[c-1] = $c;
      TimesCount = NF-2;
      next;
   }
   {
      cnt_total = $(NF-1)+0;
      avg_total = $NF+0;
      if (avg_total < Threshold) next;
      svcname   = $1;
      out       = "";
      field     = 3;
      for (c=1; c<TimesCount; c++) {
          cnt = $(field-1);
          avg = $field;
          if (avg > Threshold) out = sprintf("%s %s (%s/%s)", out, Times[c], cnt, avg);
          field += 2;
      }
      if (out) printf "%s %s  Avg %s/%s\n",  svcname, out, cnt_total, avg_total;
   }
' deo.txt

Output:

svc300  2a-3a (78/16622.60)  Avg 4100/316.27

Please show us the required output for the following sample datas :

SVCNAME,12m-1a,1a-2a,2a-3a,3a-4a,5a-6a,6a-7a,7a-8a,8a-9a,9a-10a,10a-11a,11a-12n,12n-1p,13p-14p,14p-15p,15p-16p,16p-17p,17p-18p,18p-19p,19p-20p,20p-21p,21p-22p,22p-23p,11p-12m, TOTALS
svc100,145/0.04,124/0.4,72/0.04,2/0.14,2/0.18,67/0.4,299/0.04,392/0.04,419/0.04,497/0.04,418/0.04,705/0.03,831/0.02,118/0.04,149/0.04,109/0.04,113/0.04,44/0.04,56/0.04,74/0.04,16/0.04,23/0.04,112/0.4,       4787/0.3
svc200,159/0.07,166/0.09,89/0.10,6/0.06,0/0.00,33/0.11,250/0.12,361/0.08,344/0.09,592/0.07,480/0.07,422/0.08,222/0.09,126/0.07,118/0.07,148/0.07,202/0.05,86/0.05,88/0.05,76/0.05,18/0.06,15/0.08,121/0.06,  4122/0.08
svc300,157/0.04,154/0.04,78/16622.60,4/0.03,4/0.06,41/0.04,197/0.06,328/0.04,317/0.04,539/0.04,478/0.04,402/0.04,227/0.04,148/0.04,156/0.04,185/0.04,242/0.03,111/0.03,120/0.03,82/0.03,17/0.04,17/0.03,96/0.03,  4100/316.27

Jean-Pierre.