Inserting a field without disturbing field separator on other fields

Hi All,

I have the input as below:

cat input
032016002  2.891 97.109 16.605 27.172 24.017 32.207  0.233  0.021 39.810  0.077  0.026 19.644 13.882  0.131 11.646  0.102 11.449 76.265 23.735 16.991 83.009  8.840 91.160  0.020 99.980 52.102 47.898 44.004 55.996 39.963 18.625  0.121  1.126 40.189 91.289  2.822  0.317  5.571

I need to add a value at 19th field without disturbing the field separator between the fields.

The field separator in my input is space. and each field contains 6 positions in it except the first field.

Now the value i need to add at 19th field is below:

cat bb
 2.979

My output should look like below:

cat output
032016002  2.891 97.109 16.605 27.172 24.017 32.207  0.233  0.021 39.810  0.077  0.026 19.644 13.882  0.131 11.646  0.102 11.449  2.979 76.265 23.735 16.991 83.009  8.840 91.160  0.020 99.980 52.102 47.898 44.004 55.996 39.963 18.625  0.121  1.126 40.189 91.289  2.822  0.317  5.571

I have tried the below codes:

nawk 'NR==FNR{A[FNR]=$0; next} {$19=A[FNR] OFS $19}1' bb input > op

nawk '(getline p<f)>0{$19=p OFS $19}1' f=bb input > op2

I have got the same output from the above two codes as below:

cat op
032016002 2.891 97.109 16.605 27.172 24.017 32.207 0.233 0.021 39.810 0.077 0.026 19.644 13.882 0.131 11.646 0.102 11.449  2.979 76.265 23.735 16.991 83.009 8.840 91.160 0.020 99.980 52.102 47.898 44.004 55.996 39.963 18.625 0.121 1.126 40.189 91.289 2.822 0.317 5.571

Here in the above output, if we observe,the bb value got inserted at 19th field in the input file. But the spaces got removed from the fields which contains a combination of blank space and values.
For ex:- see field2 . field2 = 2.891 but after inserting bb value the leading blank space at field2 got removed. Likewise blankspaces got removed in other fields as well.

Any idea how to overcome this ?

Thanks in advance,
am24

Is that one of your fixed field width files again? Then you shouldn't say the field separator is space...
Your problem is ( man awk : )

Try adapting the method to copy the line into an array, then add the array element to be inserted, and print the array using a fixed width format.

1 Like

Hi Rudi,

Thanks for the reply. I will try on this and will let you know.

Regards,
am24

---------- Post updated 05-12-16 at 02:52 AM ---------- Previous update was 05-11-16 at 03:52 AM ----------

Hi Rudi,

I have tried the below code.

nawk '
              {CNF = (length()-10)/7
               printf "%9s", substr ($0,1,9)
               for (i=0;i<=CNF;i++) T[i+2] = substr ($0, 10+i*7, 7)
               T[19] = 2.979
               for (i=2; i<=CNF+2; i++) printf "%7s", T
               printf RS
              }
       ' input > ruop

with the above code, i can keep the 19th field value as 2.979 and the field width also adjusted properly. But the problem is the original value of 19th field in my input 76.265 is getting removed. Al other field values are coming correctly.

I just need to insert 2.979 in 19th filed but i should not remove any other field values. How can i overcome this ?

Could you please help me on this ?

Thanks in advance,
am24

You can modify your for loop as below.

for (i=2; i<=CNF+2; i++) {printf "%7s", T; if ( i == 19) {printf "%7s", "2.979"}}
1 Like

Hi pravin27,

Thanks for the reply. I have modified the loop as you suggested. In the output , 2.979 placed in 20th field and 76.265 placed in 19th field .

So i just modified as below:

for (i=2; i<=CNF+2; i++) {printf "%7s", T; if ( i == 18) {printf "%7s", " 2.979"}}

Now the value 2.979 is placed in 19th field and 76.265 in 20th field . and all other fields placed correctly.

Also i have question that , if 2.979 value is stored in one variable, then can i assign it to T[19] ?

Reason for this is, the 2.979 is not constant value, i am taking the value from some other file. So if i pull out the value from the file and store it in one variable then can i assign the variable value to array element ?

I have tried something on this but did not get the proper result.

Thanks in advance,
am24

You want to insert that value, i.e. it will be $19, $19 will become $20, etc.? Then you need to shove every single value one index back. Like for (i=CNF; i>18; i--) $(i+1) = $i .

IF there's NO empty fields in your file, you could extend the 19th field like $19 = 2.979 " " $19 , and then split $0 into T for formatted output.

EDIT: Of course you can use a variable for the value to be inserted: $19 = var " " $19 ! But - it needs to be an awk variable, NOT a shell variable.

1 Like

Thanks Rudi,

I will try on this and will let you know.

Regards,
am24

---------- Post updated 05-13-16 at 02:12 AM ---------- Previous update was 05-12-16 at 03:30 AM ----------

Hi Rudi,

I have tried different ways to assign variable value to array element but i did not succeed in this. for ex see the below:

nawk '{print " " $NF}' line2 > nawk6
a="/export/home/batham51/radio/nawk6"
     
     
      nawk -v '
              var="$a"
              {CNF = (length()-10)/7
              printf "%9s", substr ($0,1,9)
              for (i=0;i<=CNF;i++) T[i+2] = substr ($0, 10+i*7, 7)
              for (i=2; i<=CNF+2; i++){printf "%7s", T; if( i == 18){printf "%7s", var }}
              printf RS
             }
      ' input > ruop

Can you please help me with this ?

cat nawk6
 2.979
cat input
032016002  2.891 97.109 16.605 27.172 24.017 32.207  0.233  0.021 39.810  0.077  0.026 19.644 13.882  0.131 11.646  0.102 11.449 76.265 23.735 16.991 83.009  8.840 91.160  0.020 99.980 52.102 47.898 44.004 55.996 39.963 18.625  0.121  1.126 40.189 91.289  2.822  0.317  5.571

I need to insert nawk6 value at 19th field in my input

Thanks in advance,
am24

You can use getline in awk to get the desire output. i have modified your code as below.

nawk '{print " " $NF}' line2 > nawk6
      nawk '
              {CNF = (length()-10)/7
              printf "%9s", substr ($0,1,9)
              for (i=0;i<=CNF;i++) T[i+2] = substr ($0, 10+i*7, 7)
              for (i=2; i<=CNF+2; i++){printf "%7s", T; if( i == 18){ getline var < "/export/home/batham51/radio/nawk6"; printf "%7s", var }}
              printf RS
             }
      ' input > ruop
1 Like

You can't import shell variables into awk the way you do in above post. Please read man awk and/or many threads in here about how to use the -v option to pass variables to awk .

In your case, with one single value in file nawk6, that could be done like

Try

awk 'NR == 1 {var = $0; next}
.
.
.
' nawk6 input
2 Likes

Hi Rudi and pravin27,

Thanks for the reply.I have tried both of your codes and the both worked fine.

Now i am applying the same code on seven input files where as my nawk6 contains 7 lines in it. i can do that but i am failed to take the output to seben different output files.

my nawk6 looks like this:

cat nawk6
 2.979
 3.078
 2.980
 1.338
 3.272
 5.768
 2.827

i want to insert each value of nawk6 at the 19th field in each of my seven input(1,2,3,4,5,6,7) files and take the output to 7 different output files.

can you please help me with this?

Thanks in advance,
am24

---------- Post updated at 09:02 AM ---------- Previous update was at 08:34 AM ----------

Hi,

Since my input files are containing only 1 line. I have done it in the below way.

nawk '{print " " $NF}' 7lines > nawk7
      2       nawk '
      3               {CNF = (length()-10)/7
      4               printf "%9s", substr ($0,1,9)
      5               for (i=0;i<=CNF;i++) T[i+2] = substr ($0, 10+i*7, 7)
      6               for (i=2; i<=CNF+2; i++){printf "%7s", T; if( i == 18){ getline var < "/export/home/batham51/radio/nawk7"; printf "%7s", var }}
      7               printf RS
      8              }
      9       ' input1 input2 input3 input4 input5 input6 input7  > final
     10
     11 sed -n "1p" final > output1;sed -n "2p" final > output2;sed -n "3p" final > output3;sed -n "4p" final > output4;sed -n "5p" final > output5;sed -n "6p" final > output6;sed -n "7p" final > output7

So i got the desired results. If you have any better idea than this please let me know.

Thanks in advance,
am24

If you're very sure there's no empty fields in your file, try

awk '
FNR == NR       {P[NR] = $0
                 next
                }
FNR == 1        {FN++
                }
                {printf "%s", $1 > ("output" FN)
                 $19 = P[FN] FS $19
                 $0 = $0
                 for (i=2; i<=NF; i++) printf "%7s", $i > ("output" FN)
                 printf RS > ("output" FN)
                }

' nawk7 file1 file1 file1 file1 file1
cf out*
output1:
032016002  2.891 97.109 16.605 27.172 24.017 32.207  0.233  0.021 39.810  0.077  0.026 19.644 13.882  0.131 11.646  0.102 11.449  2.979 76.265 23.735 . . . 
output2:
032016002  2.891 97.109 16.605 27.172 24.017 32.207  0.233  0.021 39.810  0.077  0.026 19.644 13.882  0.131 11.646  0.102 11.449  3.078 76.265 23.735  . . . 
output3:
032016002  2.891 97.109 16.605 27.172 24.017 32.207  0.233  0.021 39.810  0.077  0.026 19.644 13.882  0.131 11.646  0.102 11.449  2.980 76.265 23.735  . . . 
output4:
032016002  2.891 97.109 16.605 27.172 24.017 32.207  0.233  0.021 39.810  0.077  0.026 19.644 13.882  0.131 11.646  0.102 11.449  1.338 76.265 23.735  . . . 
output5:
032016002  2.891 97.109 16.605 27.172 24.017 32.207  0.233  0.021 39.810  0.077  0.026 19.644 13.882  0.131 11.646  0.102 11.449  3.272 76.265 23.735  . . . 
1 Like

As long as there is one line in 7lines for each input file, the values in the file 7lines fall in the range -99.999 through 999.999 , the value on one line in 7lines is to added between the 18th and 19th field in each line in the corresponding input file (as long as there is one or more lines in each input file), the length of the 1st field on each line in an input file is a constant width within that input file, and the length of each field (other than the 1st field) in every input file is 7; the following awk script doesn't need you to run an awk script to create the awk6 file, nor run sed scripts.

nawk '
NR == FNR {
	f19[NR] = $1
	next
}
FNR == 1 {
	if(fc++)
		close(ofn)
	len = length($1) + 17 * 7
	ofn = "output" fc
}
{	printf("%s%7.3f%s\n", substr($0, 1, len), f19[fc],
		substr($0, len + 1)) > ofn
}' 7lines input[1-7]
1 Like

Hi Rudi and Don,

Thanks for your time on this.Both of your codes worked perfectly fine.

Can you both please explain me the code ?

Regards,
am24

awk '
FNR == NR       {P[NR] = $0                                     # collect to be inserted values from first file into P array  
                 next
                }
FNR == 1        {FN++                                           # increment file number with every new file
                }
                {printf "%s", $1 > ("output" FN)                # print first element with its own format to respective output file
                 $19 = P[FN] FS $19                             # insert respective value from P array in front of field 19
                 $0 = $0                                        # recompute NF
                 for (i=2; i<=NF; i++) printf "%7s", $i > ("output" FN)
                                                                # print all fields with constant format to output file
                 printf RS > ("output" FN)                      # print line terminator
                }
' nawk7 file1 file1 file1 file1 file1
1 Like

And here is a commented version of my code:

nawk '	# Start nawk script.
# The following section of code is processed when we are reading the 1st input
# file (i.e., when the current number of lines read (NR) is the same as the
# current number of lines read from the current input file (FNR)).
NR == FNR {
	f19[NR] = $1 	# Gather the values that are to be inserted as the new
			# field #19 in subsequent files.
	next		# Read the next input line without processing the
			# remaining steps in this script.
}

# The following section of code is processed when we see the 1st line in any
# subsequent file...
FNR == 1 {
	# Increment a count of the number of files seen...
	if(fc++)
		# and, if it is not the first one, close the previous output
		# file.
		close(ofn)
	# Calculate the the number of characters in the current input line that
	# appear before the spot where the new field is to be added.  It is
	# assumed that the length of the first field in each line is a constant
	# within any input file (which is true by definition when input files
	# contain only one line).
	len = length($1) + 17 * 7
	# Set the output file name to be used when processing this input file.
	ofn = "output" fc
}
# The following section is processed for each line in the current file.
{	# Print the 1st len characters of the current input line, followed by
	# the value to be inserted as field #19 (as a 7 character floating
	# point value with three digits after the decimal point) in this file,
	# followed by the remainder of the current input line to the current
	# output file.
	printf("%s%7.3f%s\n", substr($0, 1, len), f19[fc],
		substr($0, len + 1)) > ofn
}' 7lines input[1-7]
# The above line terminates the nawk script and names the file containing the
# values to be inserted (which much be first) and the names of the input files
# to be modified in the order you want them to be processed.

Let me know if anything needs further explanation.

1 Like

Hi Don and Rudi,

Thanks for your time and Thank you so much for a clear explanation on the code.

Regards,
am24