Replace whole lines in the file from different files

Shell Programming and Scripting
1

Hi all,
I have two parameter blocks in a configuration file, file1.conf.
First parameter block starts with PWAT-PARM1 and
ends with END PWAT-PARM1.

Second parameter block starts with PWAT-PARM2 and
ends with END PWAT-PARM2 (please see below in file1.conf).
file1.conf is fixed width columns and numeric values are written in fortran format.

I don't know how to update numeric values in file1.conf from two different files, file2.txt and file3.txt for each iteration.
Thus, numeric values in the first parameter block (PWAT-PARM1) should be updated from file2.txt
and, numeric values in the second parameter block (PWAT-PARM2) should be updated from file3.txt

<file1.conf>

  PWAT-PARM1
*** < PLS>    PETMAX    PETMIN    INFEXP    INFILD    DEEPFR    BASETP    AGWETP
*** x  - x   (deg F)   (deg F)
  101  606       40.       35.        2.        2.       0.1      0.02        0.
  END PWAT-PARM1

  PWAT-PARM2
*** <PLS >     CEPSC      UZSN      NSUR     INTFW       IRC     LZETP
*** x -  x      (in)      (in)                       (1/day)
  101  606       0.1     1.128       0.2      0.75       0.5       0.1
  END PWAT-PARM2

<file2.txt>

  101  606       20.       15.        1.        1.       0.1      0.12        0.  
  101  606      1.2.       25.        3.      0.5.       0.5      0.22        0.

<file3.txt>

  101  606       0.2     1.376       0.1      0.25       0.1       1.1 
  101  606        1.     12.76       3.1       1.0       1.1       2.1

After first iteration, for example, file1.conf should look like this.

  PWAT-PARM1
*** < PLS>    PETMAX    PETMIN    INFEXP    INFILD    DEEPFR    BASETP    AGWETP
*** x  - x   (deg F)   (deg F)
  101  606       20.       15.        1.        1.       0.1      0.12        0. 
  END PWAT-PARM1

  PWAT-PARM2
*** <PLS >     CEPSC      UZSN      NSUR     INTFW       IRC     LZETP
*** x -  x      (in)      (in)                       (1/day)
  101  606       0.2     1.376       0.1      0.25       0.1       1.1 
  END PWAT-PARM2

Seems to me, sed or awk would be a solution, but I don't know how to solve it.
I searched this forum using key word "replace lines" and "replace pattern match", but no luck.
Any help would be greatly appreciated.

Thanks,

Jae

2

Try this:

awk 'BEGIN{getline v1 < "file2.txt";getline v2 < "file3.txt"}
/PWAT-PARM1/{print;getline;print;getline;print;print v1;next}
/PWAT-PARM2/{print;getline;print;getline;print;print v2;next}
{print}
' file1

Regards

3

Hi Franklin,

Thanks to you, I am almost got the solution, but I don't know how to loop in getline from two files. Only first line from file2.txt and file3.txt is updated in file.conf. For next iteration, file.conf should be updated with second line from these two files.

Jae,

awk 'BEGIN{getline v1 < "file2.txt";getline v2 < "file3.txt"}
/PWAT-PARM3/{print;getline;print;getline;print;getline
        if (match($0,match(v1,$0))) {
                        print v1;
                        getline
                } else {
                        # do some stuff here
                        print
                }
			next
			}
/PWAT-PARM4/{print;getline;print;getline;print;getline
        if (match($0,match(v2,$0))) {
                        print v2;
                        getline
                } else {
                        # do some stuff here
                        print
                }
			next
			}
{print}
' file1
4

why does it have to be in separate iterations?
why not grab all the lines from file1.txt and file2.txt immediately?

oh and another question:

why would PARM1 match-up with file2.txt?
why not match the numbers?

5

anyways... the problem is 2-fold. I think what you want is a bunch of
different configuration files in order to do some serious testing.

I don't believe you even need an original configuration file.

So, if I'm understanding you correctly:

You have a list of 2 different sets of data that are required for a single configuration.
Each configuration should contain one line from each set of data.
So in the end, you should have as many configuration files as you have lines of data.

To that end, here's the solution:

#----------------------------------------------------------------------#
# Internal field separator to pipe.                                    #
#----------------------------------------------------------------------#
IFS="|"

#----------------------------------------------------------------------#
# Counter for file name.                                               #
#----------------------------------------------------------------------#
num=1

#----------------------------------------------------------------------#
# Paste parameter files together with pipe symbol.                     #
#----------------------------------------------------------------------#
paste -d\| file2.txt file3.txt |
while read a b ; do

#----------------------------------------------------------------------#
# Create numbered configuration files.                                 #
#----------------------------------------------------------------------#
  cat << EOF > file$num.conf
PWAT-PARM1
*** < PLS> PETMAX PETMIN INFEXP INFILD DEEPFR BASETP AGWETP
*** x - x (deg F) (deg F)
$a
END PWAT-PARM1

PWAT-PARM2
*** <PLS > CEPSC UZSN NSUR INTFW IRC LZETP
*** x - x (in) (in) (1/day)
$b
END PWAT-PARM2
EOF

#----------------------------------------------------------------------#
# Next number...                                                       #
#----------------------------------------------------------------------#
  (( num += 1 ))

done

This script can be easily modified for any type of configuration file,
and any number of parameters.

6

Hi quirkasaurus,

Your code is also attractive and looks fine. But, I don't know why I got error message at the end of file.
"./file.sh: line 35: syntax error: unexpected end of file"

Any suggestions?

Jae

7

did you miss the "done" command when copying and pasting?

if not.... post of all the script here and we'll see what's up.

8

Here is the code from notepad++.
Windows-based editor create a problem?

file2.txt

  101  606       20.       15.        1.        1.       0.1      0.12        0.
  101  606      1.2.       25.        3.      0.5.       0.5      0.22        0.

file3.txt

  101  606       0.2     1.376       0.1      0.25       0.1       1.1 
  101  606        1.     12.76       3.1       1.0       1.1       2.1

shell script

#----------------------------------------------------------------------#
# Internal field separator to pipe.                                    #
#----------------------------------------------------------------------#
IFS="|"
#----------------------------------------------------------------------#
# Counter for file name.                                               #
#----------------------------------------------------------------------#
num=1
#----------------------------------------------------------------------#
# Paste parameter files together with pipe symbol.                     #
#----------------------------------------------------------------------#
paste -d\| file2.txt file3.txt |while read a b ; do
#----------------------------------------------------------------------#
# Create numbered configuration files.                                 #
#----------------------------------------------------------------------#
  cat << EOF > file$num.conf
PWAT-PARM1
*** < PLS> PETMAX PETMIN INFEXP INFILD DEEPFR BASETP AGWETP
*** x - x (deg F) (deg F)
$a
END PWAT-PARM1

PWAT-PARM2
*** <PLS > CEPSC UZSN NSUR INTFW IRC LZETP
*** x - x (in) (in) (1/day)
$b
END PWAT-PARM2
EOF

#----------------------------------------------------------------------#
# Next number...                                                       #
#----------------------------------------------------------------------#
  (( num += 1 ))
done
9

it might have been the editor. but i did forget the first line. on my system
the default shell is /bin/ksh.... so for you....

add:

#!/bin/ksh

to the very first line.

vi the file in unix and verify that no CONTROL-M's show up.

also try changing:

(( num += 1 ))

to:

$(( num = num + 1 ))

everything else should be fine.