File splitting issues

Shell Programming and Scripting
1

Hi,

I have to spilt around 120 files based on some id .These ids are provided by an input file here.For each of the 120 files, the script will read the list of ids from input file, grep and split the files .

1) I am using grep to split the files. But, it is creating 0 byte files if the id is not present in a file.

2)Also, the original file conatins table column names as the first row. This first row is different for all the 120 files. How do I add this heading row to the split files?

3) How can I use 'awk' instead of grep here? Which is faster?

The code is as below -

for pharmacyf in *
do
tablename=` cut �f3 -d'.' $pharmacyf `
 
while read pharmacyid
do
grep -w $pharmacyid $pharmacyf>> $POSOUT/ODS.POS.$pharmacyid.$tablename.$CURRENT_DATE
 
done<inputfile
done

Thanks
Maya

2

could you please list the format of your pharmacy and pharmacy id files?

3

Pharmacy id file is just a text file - each line contains one pharmacyid

Pharmacy files are created from different tables. Each file format is different. One sample is attached.

Thanks
Maya

4

I cannot understand your requirement clearly.. Instead of saying input file it would be better to mention the exact name of file you use. Also it would be better to have the few possible values of "*" in your "for pharmacyf in *".

If you dont want GREP to create a 0 byte if there is no match, then use like this

str=`grep -w $pharmacyid $pharmacyf`
if [ $? -eq 0 -a  ! -z ${str} ]
then 
  echo "${str}" >> $POSOUT/ODS.POS.$pharmacyid.$tablename.$CURRENT_DATE
fi

you can add this with in your while loop

5

Thanks Ramkrix.

I will explain the requirement here.

I need to split a list of files (around 120) in a folder based on pharmacy id. Each of these files (ODS.POS.$pharmacyid.$tablename.$CURRENT_DATE
)is created from different tables and contain table column name as header(the first line of the file). The pharmacyids are provided as an input file(in the code mentioned as inputfile). After splitting, each new file should also contain the table column name as header.

This code is not creating the header for the new files.

for pharmacyf in *
do
tablename=` cut �f3 -d'.' $pharmacyf ` 
while read pharmacyid
do
grep -w $pharmacyid $pharmacyf>> $POSOUT/ODS.POS.$pharmacyid.$tablename.$CURRENT_DATE 
 
done<inputfile
done

Hope the requirement is clear now. Performance is a major criteria here. We need to run a set of scripts in 20 mts before the next team pick it up.

Thanks
Maya

6

Hi

for pharmacyf in *
do
tablename=` cut -f3 -d'.' $pharmacyf ` 
while read pharmacyid
do
FILE="$POSOUT/ODS.POS.$pharmacyid.$tablename.$CURRENT_DATE"
[ -f  $FILE ] || echo "---header info---"  >$FILE
grep -w $pharmacyid $pharmacyf>> $POSOUT/ODS.POS.$pharmacyid.$tablename.$CURRENT_DATE 
 
done<inputfile
done

Not tested though...

Guru.