shell script to join multiple files

Shell Programming and Scripting
1

I am a new to Linux and try to write a script to join three multiple files.

For example, there are three files

file1
# comment
a Kevin
b Vin
c Sam

file 2
# comment
a 10
b 20
c 40

file 3
# comment
a blue
b yellow
c red

I am trying to ignore the line starting with #, and join only three lines. but because of the comment I cannot use join function.
Can anybody help me??

2

The "cat" command concatenates multiple files.
The comment can be removed in a variety of ways, one of which is by using sed.

$          
$ cat file1
# comment  
a Kevin    
b Vin      
c Sam      
$          
$ cat file2
# comment  
a 10       
b 20
c 40
$
$ cat file3
# comment
a blue
b yellow
c red
$
$ cat file1 file2 file3
# comment
a Kevin
b Vin
c Sam
# comment
a 10
b 20
c 40
# comment
a blue
b yellow
c red
$
$ cat file1 file2 file3 | sed '/^#/d'
a Kevin
b Vin
c Sam
a 10
b 20
c 40
a blue
b yellow
c red
$
$

Hope that helps,
tyler_durden

3 
$ cat file1 file2 file3 | sed '/^#/d'
a Kevin
b Vin
c Sam
a 10
b 20
c 40
a blue
b yellow
c red
$
$

like your code outputs, the output is


a Kevin
b Vin
c Sam
a 10
b 20
c 40
a blue
b yellow
c red

can we make it to like

a Kevin 10 blue
b Vin 20 yellow
c Sam 40 red

so it joins together??

4

try this :

$ awk '$0 !~ /#/{arr[$1]=arr[$1] " " $2}END{for(i in arr)print i,arr[i]}' file1
file2 file3 | tr -s ' '

cheers,
Devaraj Takhellambam

5

If you were going to use the join command it would be a two step operation. The awk solution that Devaraj provided is much better.

echo "
# comment
a Kevin
b Vin
c Sam
" > test1

echo "
# comment
a 10
b 20
c 40
" > test2

echo "
# comment
a blue
b yellow
c red
" > test3


join -1 1 -2 1 -t" " -o "1.1,1.2,2.2" test1 test2 > test4

join -1 1 -2 1 -t" " -o "1.1,1.2,1.3,2.2" test4 test3 | sed '/^#/d'
6

Thanks those join and awk function helps me a lot.

Just curious, what happens if file1 has

# comment
a, Kevin
b, Vin
c, Sam

what I am trying to do now is using tr to remove ',' and use join commands. Is there any better way ???

7

If file1 is comma seperated.

awk '$0 ~ /,/{gsub(/,/," ",$0)}$0 !~ /#/{arr[$1]=arr[$1] " " $2}END{for(i in arr)print i,arr[i]}' file1 file2 file3 | tr -s ' '

cheers,
Devaraj Takhellambam

8

this looks like homework for me... i' close this thread!