Copy a field into n line in another place

Shell Programming and Scripting
1

I have a file contains 100 section (here i show 2 section of that); i want to replace date from 1st & 2nd field of 1st line of each section (1st line of each section is header and the other lines are body) and merg them into one field, then copy it on 7th field of the lines in each section separately ( make a new field with date value ). after this editing, i want to delete whole 1st line (header) and also delete 2nd 4th 8th 9th 10th and 11th field of each section and put "?" instead some of them.[each section separate with ### ]
[in_file]:

 2006 0101 1236 49.3 L  37.902  48.482  0.0  Teh  5 0.2 2.7LTeh                1
 BST  SN EPn  0   1236 45.64                                     0.0    0.0   0 
 HSH  SZ EPg  0   1237 9.75                                     -0.0    126 238 
 BST  SZ EPn  4   1237 10.80                                    -3.6    143 261
###
 2006 0101 1316 22.9 L  34.933  46.292  2.0  Teh  4 0.3 2.2LTeh                1
  DHR  SZ EPg  0   1316 27.32                                    -0.1   27.0 161 
 DHR  SN ESg  0   1316 30.52                                     0.0   27.0 161 
 LIN  SZ EPg  0   1316 32.78                                    -0.1   61.0  91 
 GHG  SZ EPg  0   1316 34.96                                     0.4   72.0 159

[desired out_file]:

BST ? ? ? Pn ? 20060101 1236 45.64
HSH ? ? ? Pg ? 20060101 1237 9.75 
BST  ? ? ? Pn ? 20060101 1237 10.80
##
DHR  ? ? ? Pg  ? 20060101  1316 27.32
DHR  ? ? ? Sg  ? 20060101  1316 30.52
LIN    ? ? ? Pg  ? 20060101  1316 32.78 
GHG  ? ? ? Pg  ? 20060101  1316 34.96
2

here's a quickie that may do what you want... add error checking and adjust the regex to more exactly match your input file... don't know what your data represents so forgive the single-char variable names.

#!/bin/perl

open(I, "< in_file");
open(O, "> out_file");
$inbody=0;
while(<I>) {
    chomp;
    next if /^\s*$/;
    if (/^###/) {
        $inbody=0;
        print O "$_\n";
        next;
    }
    if ($inbody eq 0) {
        ($yr, $md) = /^\s*(\d+)\s+(\d+)/;
        $inbody=1;
    } else {
        ($a, $b, $c, $d) = /^\s*(\S+)\s*\S+\s+(\S+)\s+\S+\s+(\S+)\s+(\S+)/;
        print O "$a ? ? ? $b ? ${yr}${md} $c $d\n";
    }
}
close(I);
close(O);
1 Like
3

thank you so much, but could you please write an awk, csh or bash script to do that?

4

Why can't you use perl oreka18?

1 Like
5

because i don't writing scripts with this function, I'm learning awk.

6

alright, try:

awk '!n{dd=$1 $2; n=1; next} {sub($2 FS substr($3,1,1),"? ? ? "); $4="? " dd; NF=8} /^###/{$0=$1; n=0}1' infile
1 Like
7

try this :wink:

# awk '/^ 200?/{x=$1$2;y="? ? ?";next}{sub(/^./,"",$3);if(/^##/)print "##";else print $1,y,$3,"?",x,$5,$6}' yourfile
BST ? ? ? Pn ? 20060101 1236 45.64
HSH ? ? ? Pg ? 20060101 1237 9.75
BST ? ? ? Pn ? 20060101 1237 10.80
##
DHR ? ? ? Pg ? 20060101 1316 27.32
DHR ? ? ? Sg ? 20060101 1316 30.52
LIN ? ? ? Pg ? 20060101 1316 32.78
GHG ? ? ? Pg ? 20060101 1316 34.96
1 Like
8

thanks alot but there is a little problem:
the output of your script contains the values of 1st field (header must be deleted):

? ? ?  ? 20060101  
GHG ? ? ?  ? 20060101 1316 44.78
LIN ? ? ?  ? 20060101 1316 42.60

i want this output

GHG ? ? ?  ? 20060101 1316 44.78
LIN ? ? ?  ? 20060101 1316 42.60
9

add to an asteriks and try again

# awk '/^ *200?/........

so

# awk '/^ *200?/{x=$1$2;y="? ? ?";next}{sub(/^./,"",$3);if(/^##/)print "##";else print $1,y,$3,"?",x,$5,$6}' yourfile

or

/usr/xpg4/bin/awk ....

or

nawk ...
1 Like
10

I get:

BST ? ? ? 20060101 Pn 0 1236 45.64
HSH ? ? ? 20060101 Pg 0 1237 9.75
BST ? ? ? 20060101 Pn 4 1237 10.80
###
DHR ? ? ? 20060101 Pg 0 1316 27.32
DHR ? ? ? 20060101 Sg 0 1316 30.52
LIN ? ? ? 20060101 Pg 0 1316 32.78
GHG ? ? ? 20060101 Pg 0 1316 34.96

Try nawk instead of awk (only guessing, I still do not know your OS and version)?

1 Like
11

oh I'm sorry, i forget to tell to you. it's fedora 13.
actually in my original file there is no "###" between section maybe it occur such that problem

12

i tried in fc14 :wink: is it ok ?

# awk '/^ *200?/{x=$1$2;y="? ? ?";next}!/^ *$/{sub(/^./,"",$3);print $1,y,$3,"?",x,$5,$6;}' yourfile
BST ? ? ? Pn ? 20060101 1236 45.64
HSH ? ? ? Pg ? 20060101 1237 9.75
BST ? ? ? Pn ? 20060101 1237 10.80
DHR ? ? ? Pg ? 20060101 1316 27.32
DHR ? ? ? Sg ? 20060101 1316 30.52
LIN ? ? ? Pg ? 20060101 1316 32.78
GHG ? ? ? Pg ? 20060101 1316 34.96
1 Like
13

but my input file and output file are?
[orginal_file]

2006 0101 1236 49.3 L  37.902  48.482  0.0  Teh  5 0.2 2.7LTeh                1 
 GAP=238                                                                       E 
 Iranian Seismological Center, Institute of Geophysics, University of Tehran   6 
 STAT SP IPHASW D HRMM SECON CODA AMPLIT PERI AZIMU VELO SNR AR TRES  W DIS CAZ7 
 HSH  SZ E    0   1237 29.15      2843.4 0.04                           126 238 
 AZR  SZ E    0   1237 58.33       598.9 0.14                           221 264 
 MRD  SZ E    0   1238  5.28       615.6 0.22                           258 291 
 BST  SZ E    0   1237 31.34       897.3 0.08                           143 261 
 SHB  SZ E    0   1238  2.03       222.2 0.16                           254 279 
 AZR  SE EPn  0   1236 40.89                                     0.0    0.0   0 
 BST  SN EPn  0   1236 45.64                                     0.0    0.0   0 
 HSH  SZ EPg  0   1237 9.75                                     -0.0    126 238 
 BST  SZ EPn  4   1237 10.80                                    -3.6    143 261 
 AZR  SZ EPn  0   1237 24.31                                     0.3    221 264 
 SHB  SZ EPn  0   1237 27.99                                     0.0    254 279 
 MRD  SZ EPn  0   1237 28.80                                     0.3    258 291 
 BST  SN ELg  0   1237 29.38                                    -0.0    143 261 
 AZR  SE ELg  0   1237 51.43                                     0.0    221 264 
 
 2006 0101 1316 22.9 L  34.933  46.292  2.0  Teh  4 0.3 2.2LTeh                1 
 GAP=290                                                                       E 
 Iranian Seismological Center, Institute of Geophysics, University of Tehran   6 
 STAT SP IPHASW D HRMM SECON CODA AMPLIT PERI AZIMU VELO SNR AR TRES  W DIS CAZ7 
 GHG  SZ E    0   1316 44.78      2607.6 0.06                          72.0 159 
 LIN  SZ E    0   1316 42.60      2426.4 0.08                          61.0  91 
 DHR  SZ E    0   1316 30.90    108786.0 0.04                          27.0 161 
 KOM  SZ E    0   1317  7.20       713.4 0.18                           140 126 
 DHR  SZ EPg  0   1316 27.32                                    -0.1   27.0 161 
 DHR  SN ESg  0   1316 30.52                                     0.0   27.0 161 
 LIN  SZ EPg  0   1316 32.78                                    -0.1   61.0  91 
 GHG  SZ EPg  0   1316 34.96                                     0.4   72.0 159 
 LIN  SN ESg  0   1316 40.32                                     0.0   61.0  91 
 KOM  SZ EPg  0   1316 45.20                                    -0.4    140 126

[out_file] after do your script:

GAP=238 ? ? ?  ? 20060101  
Iranian ? ? ? enter, ? 20060101 of Geophysics,
STAT ? ? ? PHASW ? 20060101 HRMM SECON
HSH ? ? ?  ? 20060101 1237 29.15
AZR ? ? ?  ? 20060101 1237 58.33
MRD ? ? ?  ? 20060101 1238 5.28
BST ? ? ?  ? 20060101 1237 31.34
SHB ? ? ?  ? 20060101 1238 2.03
AZR ? ? ? Pn ? 20060101 1236 40.89
BST ? ? ? Pn ? 20060101 1236 45.64
HSH ? ? ? Pg ? 20060101 1237 9.75
BST ? ? ? Pn ? 20060101 1237 10.80
AZR ? ? ? Pn ? 20060101 1237 24.31
SHB ? ? ? Pn ? 20060101 1237 27.99
MRD ? ? ? Pn ? 20060101 1237 28.80
BST ? ? ? Lg ? 20060101 1237 29.38
AZR ? ? ? Lg ? 20060101 1237 51.43
  ? ? ?  ? 20060101  
GAP=290 ? ? ?  ? 20060101  
Iranian ? ? ? enter, ? 20060101 of Geophysics,
STAT ? ? ? PHASW ? 20060101 HRMM SECON
GHG ? ? ?  ? 20060101 1316 44.78
LIN ? ? ?  ? 20060101 1316 42.60
DHR ? ? ?  ? 20060101 1316 30.90
KOM ? ? ?  ? 20060101 1317 7.20
DHR ? ? ? Pg ? 20060101 1316 27.32
DHR ? ? ? Sg ? 20060101 1316 30.52
LIN ? ? ? Pg ? 20060101 1316 32.78
GHG ? ? ? Pg ? 20060101 1316 34.96
LIN ? ? ? Sg ? 20060101 1316 40.32
KOM ? ? ? Pg ? 20060101 1316 45.20

and i want this file
[desired]

HSH ? ? ?  ? 20060101 1237 29.15
AZR ? ? ?  ? 20060101 1237 58.33
MRD ? ? ?  ? 20060101 1238 5.28
BST ? ? ?  ? 20060101 1237 31.34
SHB ? ? ?  ? 20060101 1238 2.03
AZR ? ? ? Pn ? 20060101 1236 40.89
BST ? ? ? Pn ? 20060101 1236 45.64
HSH ? ? ? Pg ? 20060101 1237 9.75
BST ? ? ? Pn ? 20060101 1237 10.80
AZR ? ? ? Pn ? 20060101 1237 24.31
SHB ? ? ? Pn ? 20060101 1237 27.99
MRD ? ? ? Pn ? 20060101 1237 28.80
BST ? ? ? Lg ? 20060101 1237 29.38
AZR ? ? ? Lg ? 20060101 1237 51.43
  
GHG ? ? ?  ? 20060101 1316 44.78
LIN ? ? ?  ? 20060101 1316 42.60
DHR ? ? ?  ? 20060101 1316 30.90
KOM ? ? ?  ? 20060101 1317 7.20
DHR ? ? ? Pg ? 20060101 1316 27.32
DHR ? ? ? Sg ? 20060101 1316 30.52
LIN ? ? ? Pg ? 20060101 1316 32.78
GHG ? ? ? Pg ? 20060101 1316 34.96
LIN ? ? ? Sg ? 20060101 1316 40.32
KOM ? ? ? Pg ? 20060101 1316 45.20
14 
# awk '/^ *200?/{x=$1$2;y="? ? ?";while($1!~/STAT/)getline;next}!/^ *$/{sub(/^./,"",$3);print $1,y,$3,"?",x,$5,$6;}/^ *$/{print}' input
HSH ? ? ?  ? 20060101 1237 29.15
AZR ? ? ?  ? 20060101 1237 58.33
MRD ? ? ?  ? 20060101 1238 5.28
BST ? ? ?  ? 20060101 1237 31.34
SHB ? ? ?  ? 20060101 1238 2.03
AZR ? ? ? Pn ? 20060101 1236 40.89
BST ? ? ? Pn ? 20060101 1236 45.64
HSH ? ? ? Pg ? 20060101 1237 9.75
BST ? ? ? Pn ? 20060101 1237 10.80
AZR ? ? ? Pn ? 20060101 1237 24.31
SHB ? ? ? Pn ? 20060101 1237 27.99
MRD ? ? ? Pn ? 20060101 1237 28.80
BST ? ? ? Lg ? 20060101 1237 29.38
AZR ? ? ? Lg ? 20060101 1237 51.43

GHG ? ? ?  ? 20060101 1316 44.78
LIN ? ? ?  ? 20060101 1316 42.60
DHR ? ? ?  ? 20060101 1316 30.90
KOM ? ? ?  ? 20060101 1317 7.20
DHR ? ? ? Pg ? 20060101 1316 27.32
DHR ? ? ? Sg ? 20060101 1316 30.52
LIN ? ? ? Pg ? 20060101 1316 32.78
GHG ? ? ? Pg ? 20060101 1316 34.96
LIN ? ? ? Sg ? 20060101 1316 40.32
KOM ? ? ? Pg ? 20060101 1316 45.20

regards
ygemici

1 Like
15

thank you very much, now i want to print each line which the 5th field of that contains stings "*P,P*" or "*s,S*" {P,S,Pg,Sg,Pn,Sn} without losing blank lines.

16

i suppose that you meant the $3 coloumn (contains -> EPg,ESg) ?

#  awk --posix '{if(!/^ *$/){if($3~/EP[ng]{0,1}$/||$3~/ES[ng]{0,1}$/)print}else print}' yourfile
1 Like
17

thanks but there is no output! did you test it on the data that i leaved on previews section?

18 
# cat /etc/redhat-release
Fedora release 14 (Laughlin)

# cat yourfile
2006 0101 1236 49.3 L  37.902  48.482  0.0  Teh  5 0.2 2.7LTeh                1
 GAP=238                                                                       E
 Iranian Seismological Center, Institute of Geophysics, University of Tehran   6
 STAT SP IPHASW D HRMM SECON CODA AMPLIT PERI AZIMU VELO SNR AR TRES  W DIS CAZ7
 HSH  SZ E    0   1237 29.15      2843.4 0.04                           126 238
 AZR  SZ E    0   1237 58.33       598.9 0.14                           221 264
 MRD  SZ E    0   1238  5.28       615.6 0.22                           258 291
 BST  SZ E    0   1237 31.34       897.3 0.08                           143 261
 SHB  SZ E    0   1238  2.03       222.2 0.16                           254 279
 AZR  SE EPn  0   1236 40.89                                     0.0    0.0   0
 BST  SN EPn  0   1236 45.64                                     0.0    0.0   0
 HSH  SZ EPg  0   1237 9.75                                     -0.0    126 238
 BST  SZ EPn  4   1237 10.80                                    -3.6    143 261
 AZR  SZ EPn  0   1237 24.31                                     0.3    221 264
 SHB  SZ EPn  0   1237 27.99                                     0.0    254 279
 MRD  SZ EPn  0   1237 28.80                                     0.3    258 291
 BST  SN ELg  0   1237 29.38                                    -0.0    143 261
 AZR  SE ELg  0   1237 51.43                                     0.0    221 264

 2006 0101 1316 22.9 L  34.933  46.292  2.0  Teh  4 0.3 2.2LTeh                1
 GAP=290                                                                       E
 Iranian Seismological Center, Institute of Geophysics, University of Tehran   6
 STAT SP IPHASW D HRMM SECON CODA AMPLIT PERI AZIMU VELO SNR AR TRES  W DIS CAZ7
 GHG  SZ E    0   1316 44.78      2607.6 0.06                          72.0 159
 LIN  SZ E    0   1316 42.60      2426.4 0.08                          61.0  91
 DHR  SZ E    0   1316 30.90    108786.0 0.04                          27.0 161
 KOM  SZ E    0   1317  7.20       713.4 0.18                           140 126
 DHR  SZ EPg  0   1316 27.32                                    -0.1   27.0 161
 DHR  SN ESg  0   1316 30.52                                     0.0   27.0 161
 LIN  SZ EPg  0   1316 32.78                                    -0.1   61.0  91
 GHG  SZ EPg  0   1316 34.96                                     0.4   72.0 159
 LIN  SN ESg  0   1316 40.32                                     0.0   61.0  91
 KOM  SZ EPg  0   1316 45.20                                    -0.4    140 126

# awk --posix '{if(!/^ *$/){if($3~/EP[ng]{0,1}$/||$3~/ES[ng]{0,1}$/)print}else print}' yourfile
 AZR  SE EPn  0   1236 40.89                                     0.0    0.0   0
 BST  SN EPn  0   1236 45.64                                     0.0    0.0   0
 HSH  SZ EPg  0   1237 9.75                                     -0.0    126 238
 BST  SZ EPn  4   1237 10.80                                    -3.6    143 261
 AZR  SZ EPn  0   1237 24.31                                     0.3    221 264
 SHB  SZ EPn  0   1237 27.99                                     0.0    254 279
 MRD  SZ EPn  0   1237 28.80                                     0.3    258 291

 DHR  SZ EPg  0   1316 27.32                                    -0.1   27.0 161
 DHR  SN ESg  0   1316 30.52                                     0.0   27.0 161
 LIN  SZ EPg  0   1316 32.78                                    -0.1   61.0  91
 GHG  SZ EPg  0   1316 34.96                                     0.4   72.0 159
 LIN  SN ESg  0   1316 40.32                                     0.0   61.0  91
 KOM  SZ EPg  0   1316 45.20                                    -0.4    140 126