comparing files - adding/subtracting/formating columns

Shell Programming and Scripting
1

I have two files:
file1.txt:
FS Total Used Free Used%
/u01 10000 8000 2000 80%
/u02 10000 8000 2000 80%
/u03 10000 8000 2000 80%
/u04 10000 8000 2000 80%
/u05 10000 8000 2000 80%
/u06 10000 8000 2000 80%
/u07 10000 8000 2000 80%
/u10 10000 5000 5000 50%

file2.txt:
FS Adj
/u01 1,500
/u05 500
/u10 2,500

I would like to compare them using the first column in each file and create an output from both that looks like the following:

FS Total Used+Adj Free-Adj (Used+Adj)/Total
--------- -------- --------- --------- ---------
/u01 10,000 9,500 500 95%
/u02 10,000 8,000 2,000 80%
/u03 10,000 8,000 2,000 80%
/u04 10,000 8,000 2,000 80%
/u05 10,000 8,500 1,500 85%
/u06 10,000 8,000 2,000 80%
/u07 10,000 7,500 2,500 75%

Please note that all lines from file1.txt are listed, and column "adj" of file2.txt is added to column "Used" and subtracted from column "Free" of file1.txt only if there is a match.

I was able to produce this report only after loading these files into a database but I am sure I can do it using shell scripting with your help.

Thanks,
Omer

2

You can use something like this:
(use nawk or /usr/xpg4/bin/awk on Solaris)

awk 'NR == FNR { 
  sub(/,/, "")
  _[$1] = $2 
  next 
  }
FNR == 1 { 
  printf "%4s %5s %8s %8s %14s\n", 
  "FS", "Total", "Used+Adj", "Free-Adj", "(Used+Adj)/Total"
  while (++i < 46) printf "-"
  print ""
  next
  }           
$1 in _ { 
  $3 += _[$1]
  $4 -= _[$1] 
  $5 = $3/$2*100 
  }
{
  printf "%4s %5d %8d %8d %14d%\n",
  $1, $2, $3, $4, $5
  }' file2.txt file1.txt
3

Thank you, radoulov, for your quick and elegent script. It is working perfectly.
All I need to do now is format the report nicely so it is more readable but I think I can figure that out.

Appreciate it

Omer

4

Hi Radoulov,
I have read your script .Can you eloborate for me for knowledge .Thanks.

5

Sure.

NR == FNR

The above expression returns true only while processing the first input file (file2). So, the corresponding actions are:

{ 
  sub(/,/, "")
  _[$1] = $2 
  next 
  }

Strip the thousands separator, populate an associative array _with $1 as keys, $2 as values, so given the input it will be:

key -> "/u01" value -> 1500
key -> "/u05" value -> 500
key -> "/u10" value -> 2500

The next statement forces awk to stop processing the current record so no further rules/actions will be executed for this record. This means that the next actions won't process the first input file.

FNR == 1

FNR is the number of records that have been read so far from the current input file (the second input file in this case). So, while processing the first record of the second input file:

{ 
  printf "%4s %5s %8s %8s %14s\n", 
  "FS", "Total", "Used+Adj", "Free-Adj", "(Used+Adj)/Total"
  while (++i < 46) printf "-"
  print ""
  next
  }

Print the header columns and go to the next record.

$1 in _ { 
  $3 += _[$1]
  $4 -= _[$1] 
  $5 = $3/$2*100 
  }

The expression key in array returns true if the indicated key exists in the indicated array. For those records do the following:

  • add the value _[$1] of the corresponding key $1 to the third column
  • subtract the value _[$1] of the corresponding key $1 from the fourth column
  • calculate the value of the fifth column
{
  printf "%4s %5d %8d %8d %14d%\n",
  $1, $2, $3, $4, $5
  }

Print the new values.

Hope this helps.

6

SIMPLY SUPERB !! NO WORDSSSSSSSS WILL SUFFICE YOUR WAY OF
EXPLAINING . BUNDLES OF THANKS FROM BOTTOM OF MY HEART.Let me
say to understand is somthing and to EXPLAIN in simplicity which is more
cumbersome.Once again Thanx .
Cheers Dimitri.

7

Just now I have written the reply but appers no where,

Dear Radlouv
Bundles of Thanks to you for explaining the code in very simple term .
No way , but SIMPLY SUPERB .No word will suffice . Knowing is something
but to explain in simple wrods requires great insite .

Thanx once again. Cheers Up.

8

Glad you find it useful :slight_smile:

Regards