Advice on Script

Shell Programming and Scripting
1

I would like some advice on how to logically put together a script to handle a daily task of data gathering for the following problem.

I have two files, file1 has 125,000 records that I cut and remove unwanted fields through scripts and cron. In file2, I have 25000 records that has the same data as file1 plus one extra field of data. I would like the records from file2 (25,000) to replace entries in file1 with the extra data which would then be file3.

I hope I can express this clearly and appreicate any help.

Eample:
file1
00:01:A6:A9:3A:7E|20070103_120040|1|8|0|0|
00:0A:73:1F:13:54|20070103_120041|1|8|0|0|
00:40:7B:D6:74:E8|20070103_120041|1|8|0|0|

file2
00:01:A6:A9:3A:7E|20070103_120040|1|8|0|0|6 days
00:0A:73:1F:13:54|20070103_120041|1|8|0|0|37 day
00:40:7B:D6:74:E8|20070103_120041|1|8|0|0|24 days

file3
00:01:A6:A9:3A:7E|20070103_120040|1|8|0|0|6 days
00:0A:73:1F:13:54|20070103_120041|1|8|0|0|37 day
00:40:7B:D6:74:E8|20070103_120041|1|8|0|0|24 days
00:0A:73:25:B9:EC|20070103_120039|1|7|0|0|
00:01:A6:96:48:60|20070103_120040| | | | |
00:01:A6:A9:3A:7E|20070103_120040|1|8|0|0|
00:0A:73:1F:13:54|20070103_120041|1|8|0|0|
00:40:7B:D6:74:E8|20070103_120041|1|8|0|0|
00:0A:73:23:BA:A8|20070103_120042| | | | |

2

I did not get it clearly enough.

I would like the records from file2 (25,000) to replace entries in file1 with the extra data

records which has to be replaced in file1 with records from file2, do they have any specific criteria or is that just file2 records and replace any record from file1 to create file3.

Please clarify.

Thanks
Sumeet

3

sumeet,

File2 data (25,000) records will replace the 25,000 records in file1. The records can be in any order as long as the entire line is replaced. The only net gain is the data in the last column, field making total records remaining at 125,000.

I hope this helps.

4

#!/usr/bin/ksh

> file3
while read LINE
do
LINE2=`grep "^${LINE}|" file2 2> /dev/null`
if [ ${?} -eq 0 ]
then
echo "${LINE2} >> file3
else
echo "${LINE} >> file3
fi
done < file1

5

That would fire off a lot of grep processes. This might be faster:
sort -t'|' -k1,6 -k7r file1 file2| sort -m -t'|' -k1,6 -u > file3