Issue with search and replacing multiple items in multiple files

UNIX for Beginners Q & A
1

Im having an issue when trying to replace the first column with a new set of values in multiple files. The results from the following code only replaces the files with the last set of values in val.txt. I want to replace all the files with all the values.

for date in {1..31}
do
for val in `cat val.txt`
do
csv="file/location/csv"
fid=$(awk -F"," '$2 == "'$val'" { print $3 }' $csv)
awk '$1=='${val}'{$1='${fid}'}{print}' data-${date}.txt > data-${date}-new.txt
done
done

For instance:

val.txt
01
02
03
04

csv

abc,01,01
abc,50,02
abc,02,04
abc,03,05
abc,04,06

data-1.txt

01  001  1.062
01  003  0.216
01  005  1.89
01  007  0.828
02  001  1.44
02  003  -3.492
02  005  -4.536
03  147  -0.144
03  149  0.216
04  001  -0.144
04  003  -1.62

The result I was hoping for was this in all of the files:

data-1-new.txt

01  001  1.062
01  003  0.216
01  005  1.89
01  007  0.828
04  001  1.44
04  003  -3.492
04  005  -4.536
05  147  -0.144
05  149  0.216
06  001  -0.144
06  003  -1.62
2

Why that complicated? Try

awk '
FNR == 1        {FC++
                 FN = FILENAME
                 sub (/.txt/, "-new&", FN)
                }
FC == 1         {VAL[$1]
                 next
                }
FC == 2         {CSV[$2] = $3
                 next
                }
$1 in VAL       {$1 = CSV[$1]
                }
                {print >  FN
                }

' FS=, val.txt file/location/csv FS=" " data-*.txt

It will read the val and csv files and put their values into respective arrays, and then pick any data*.text file around and convert it into a "new" one. If you want to restrict the file selection to 1 .. 31 but skip non-exististing ones, try using shell patterns. You may need to adapt the field separator FS .

1 Like
3
Moderator comments were removed during original forum migration.