awk separate files to one directory

Shell Programming and Scripting
1

I am trying to output all files that are made by this awk to a specific directory.

 awk -F '[ :]' '{f = $3 ".txt"; print > f}' input.txt

Since the actual data has several hundred files I redirect the output (well tried to) to a directory.

awk -F '[ :]' '{f = $3 ".txt";  close($3 ".txt")} print > f}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed > /home/cmccabe/Desktop/panels/BMF
 bash: /home/cmccabe/Desktop/panels/BMF: Is a directory

So I tried (thought defining the variable may help) the below but it didn't help. Thank you :).

awk -F '[ :]' '{f = $3 ".txt";  close($3 ".txt")} print > f}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed > /home/cmccabe/Desktop/panels/BMF/"$f".txt; 
bash: /home/cmccabe/Desktop/panels/BMF: Is a directory
2

Hello cmccabe,

Following may help you in same.

awk -F '[ :]' '{f = $3 ".txt";  close($3 ".txt")} print > "/home/cmccabe/Desktop/panels/BMF/" f}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed

Thanks,
R. Singh

1 Like
3

Thank you for the code :), here is the error I get.

awk -F '[ :]' '{f = $3 ".txt";  close($3 ".txt")} print > "/home/cmccabe/Desktop/panels/BMF/" f}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed
awk: line 1: syntax error at or near print
awk: line 1: extra '}'

If I remove the close($3 ".txt")} it works great. Thank you :).

4

Hello cmccabe,

Yes, you could try below.

 awk -F '[ :]' '{f = $3 ".txt";  close($3 ".txt");print > "/home/cmccabe/Desktop/panels/BMF/" f}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed

But actually speaking you should below for same.

 awk -F '[ :]' '{f = $3 ".txt";print > "/home/cmccabe/Desktop/panels/BMF/" f;  close(f);}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed

Hope this helps.

Thanks,
R. Singh

5

Not really... There are a couple of problems here.
The standards don't define the precedence between output redirection and string concatenation in awk print commands. So, although it appears to work on your system, it isn't portable. And, if you write to one string, as in:

print > ("/home/cmccabe/Desktop/panels/BMF/" f)

you need to close that same string; so:

close(f)

is a no-op because f has never been opened. So, what you really need is something more like:

awk -F '[ :]' '
{f = "/home/cmccabe/Desktop/panels/BMF/" $3 ".txt"
 print > f
 close(f)
}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed

Or, more efficiently:

cd /home/cmccabe/Desktop/panels/BMF
awk -F '[ :]' '
{f = $3 ".txt"
 print > f
 close(f)
}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed
6

He/she'd need to add a path to the input file, then...

1 Like
7

Thanks RudiC. Yes, it should have been:

srcdir="$PWD"
cd /home/cmccabe/Desktop/panels/BMF
awk -F '[ :]' '
{f = $3 ".txt"
 print > f
 close(f)
}' "$srcdir"/BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed