shell loop and awk

Shell Programming and Scripting
1

Hello, everyone!

I merge two files using awk like following, and there are two files involved in this command.

awk 'NR==FNR{A[$1]=$0; next} $1=A[$1]' file1 FS=, OFS=',' file2 > tempfile.txt

Further, I need to deal with hundreds of files at the same time.

I want to make a loop like this:

#!/bin/bash
FILES=*
for f in $FILES
do
   awk    
done

But I don't know how to deal with files in order, cause I need to put the file name instead of file2.

2

For a tmp file for each file:

for fl in *
do
   awk 'NR==FNR{A[$1]=$0; next} $1=A[$1]' file1 FS=, OFS=',' "$fl" > "$fl".tmp
done
3

Sorry, I didn't express my need clearly.

What I want to do is as follows:

awk 'NR==FNR{A[$1]=$0; next} $1=A[$1]' file1 FS=, OFS=',' file2 > file2.tmp

awk 'NR==FNR{A[$1]=$0; next} $1=A[$1]' file2.tmp FS=, OFS=',' file3 > file3.tmp

awk 'NR==FNR{A[$1]=$0; next} $1=A[$1]' file3.tmp FS=, OFS=',' file4 > file4.tmp

.
.
.
.
.

4

What results do you get with the last post? file1 should reside in another directory as file#, or use /dir/file1 and /dir/file#

5

I will get records in all these files with the same string in $1 of each file.

6

something like,

for f in *; do
  [ -n "$last" ] && awk '...code...' "$f" "$prev" > "$f".tmp
  prev=${f}.tmp
done

edit: using bash can replace [ -n "$last" ] with [[ $last ]]

7

It seems doesn't work. The code as follows:

#!/bin/bash
for f in *;
  do
    echo $f
    [ -n "$last"] && awk -F, 'NR==FNR{A[$1]=$0; next} $1=A[$1]' "$f" FS=, OFS=, "$prev" > "$f".tmp
  prev=${f}.tmp
done
8

oh i see it. i used $prev in one spot and $last in another. sorry. usually test things first but it was just a quick logic example. seems you'll also have to add something to deal with first file..

#!/bin/bash
for f in file*; do
  if [[ $prev ]]; then
    echo "cat $f $prev > ${f}.tmp"
    prev=${f}.tmp
  else
    prev=$f
  fi
done

mute@clt:~/temp/xshang$ ./script
cat file2 file1 > file2.tmp
cat file3 file2.tmp > file3.tmp
cat file4 file3.tmp > file4.tmp
cat file5 file4.tmp > file5.tmp
1 Like
9

Nice code! It works! Thank you very much!