Delimiter appending in a data file if we receive a less columns than expected

Required No.of field = 12

Let say you got a �~� delimited input file and this file has 6 input fields and now I want to add 12-5=7 number of �~� into this input file in order to make it 12 fields

datafile can have n number of records

ex.,

a~b~c~d~12~r
a~b~c~d~12~r
a~b~c~d~12~r
a~b~c~d~12~r

output expected is

a~b~c~d~12~r~~~~~~~
a~b~c~d~12~r~~~~~~~
a~b~c~d~12~r~~~~~~~
a~b~c~d~12~r~~~~~~~

which works in korn shell

Welcome to the forum.

Any attempts / ideas / thoughts from your side?

Hello LJJ,

Welcome to forums, please use code tags for your Input_file/commands/codes as per forum rules.
Could you please try following and let me know if this helps you.

awk -F"~" '{i=NF;while(i<14){val=val?val "~":"~";i++};print $0 val;val=""}'  Input_file

Thanks,
R. Singh

Also, try:

$ cat file
a~b~c~d~12~r
a~b~c~d~12~r
a~b~c~d~12~r
a~b~c~d~12~r

$ awk -F~ 'NF < 12 { $12 = FS } 1' OFS=~ file
a~b~c~d~12~r~~~~~~~
a~b~c~d~12~r~~~~~~~
a~b~c~d~12~r~~~~~~~
a~b~c~d~12~r~~~~~~~

Some awk versions allow for

awk -F"~" 'NF=12' OFS="~" file
a~b~c~d~12~r~~~~~~
a~b~c~d~12~r~~~~~~
a~b~c~d~12~r~~~~~~
a~b~c~d~12~r~~~~~~
1 Like

Hi All thanks for the response .,

i'm bit surprised as none of the solns is producing the result is it because of the shell

trying some vague approach , please help

#The below gets the no of record delimiter in the data file
head -1 datagen.txt | awk -F'~' '{ print NF-1 }' > nrd.dat
###############33

nrd.dat was holding the no of delimiters we received in the data file

#!/bin/ksh
for i in `cat nrd.dat`; do
export df=`expr 10 - $i`
echo "$df"
export dlm='~'
echo "$dlm"
done
awk 'BEGIN { while (count++<14) string=string "~"; print string ;break}' > pt.dat

There are a number of issues with your script, but that aside, I don't see any of the provided "solns" in your code.

Are you on Solaris? If so use /usr/xpg4/bin/awk instead

1 Like

Apart from the fact that in your post#1 the spec ("No.of field = 12") doesn't match the desired result (13 fields) ALL solutions proposed produce an easily adaptable result that should satisfy your request.

Hi R.Singh ., kindly help what does {val=val?val "~":"~";i++} does ?

---------- Post updated at 08:29 PM ---------- Previous update was at 08:24 PM ----------

Dear scott ., i tried the same example to execute ., and it returned the same data output, any idea what could be issue

:/data/files/jlj > cat sample.dat
a~b~c~d~12~r
a~b~c~d~12~r
a~b~c~d~12~r
a~b~c~d~12~r
:/data/files/jlj > awk -F"~" 'NF < 12 { $12 = FS; print }' OFS="~" sample.dat
a~b~c~d~12~r
a~b~c~d~12~r
a~b~c~d~12~r
a~b~c~d~12~r
:/data/files/jlj > awk -F~ 'NF < 12 { $12 = FS; print }' OFS=~ sample.dat
a/b/c/d/12/r
a/b/c/d/12/r
a/b/c/d/12/r
a/b/c/d/12/r

Then, if you are using Solaris, refer to Scrutinizer's post. Use /usr/xpg4/bin/awk or nawk.

Hello LJJ,

Could you please go through following explanation and let me know if this helps you.

awk -F"~" '{          ##Making field separator as ~ here for each line of Input_file.
i=NF;                 ##creating a variable named i whose value is equal to value of NF, NF is out of box variable which defines number of fields in a line of Input_file.
while(i<14){          ##Initiating while loop here which will run from i value from 0 to 13 here.
  val=val?val "~":"~";##creating a variable named val whose value will be having its value to ~ on very first occurrence and will concatenate itself each time with its previous value. Where ? means if condition is TRUE then execute statement followed by it or condition is FALSE then execute statement after :
  i++                 ##incrementing the value of variable named i here.
};
print $0 val;         ##printing current line and value of variable val.
val=""                ##nullifying value of val here.
}' Input_file         ##Mentioning Input_file here.
 

Thanks,
R. Singh

1 Like
/data/files/jlj > uname -a
SunOS ux562z3 5.10 Generic_150400-17 sun4u sparc SUNW,SPARC-Enterprise

---------- Post updated at 08:43 PM ---------- Previous update was at 08:39 PM ----------

Hey ., Thanks Scott .,

it worked ! it worked !

/data/files/jlj > nawk -F"~" 'NF < 12 { $12 = FS; print }' O
a~b~c~d~12~r~~~~~~~
a~b~c~d~12~r~~~~~~~
a~b~c~d~12~r~~~~~~~
a~b~c~d~12~r~~~~~~~

Thank u so much

Cool! As RudiC pointed out, what you asked for will actually give you thirteen fields. You want to reduce those numbers by 1 to get the twelve you desired (or use RudiC's solution, which gives the desired result already (although this didn't work for me, it might for you - but on Solaris, who knows? :))).

A slightly different way that should work with all versions of (n)awk :

awk '{$12=$12}1' FS=\~ OFS=\~ file

--
or, more general:

awk -v n=12 '{$n=$n}1' FS=\~ OFS=\~ file
1 Like

Thanks for the response it works for the '~' delimiter ., but for '|' this is appending from the front replacing the characters .

/data/files/jlj > nawk -v n=12 '{$n=$n}1' FS=\| OFS=\| dtd.dat > p.dat
/data/files/jlj > cat p.dat
|||||||Brennan|Shaine|Lane Avila|Graham@dignissim.org|658
|||||||bles|Nyssa|Hop Montoya|Hermione@adipiscing.com|769
||||||| Mckinney|Jana|Ashely Perkins|Meghan@rhoncus.com|522
|||||||Mccarthy|Gage|Kiara Solomon|Cathleen@ornare.us|462

Without showing us what input you had that produced that output, we have to assume that your input had leading vertical bars. When I try the code you showed us in post #16 with the input:

Brennan|Shaine|Lane Avila|Graham@dignissim.org|658
bles|Nyssa|Hop Montoya|Hermione@adipiscing.com|769
 Mckinney|Jana|Ashely Perkins|Meghan@rhoncus.com|522
Mccarthy|Gage|Kiara Solomon|Cathleen@ornare.us|462

the output I get is:

Brennan|Shaine|Lane Avila|Graham@dignissim.org|658|||||||
bles|Nyssa|Hop Montoya|Hermione@adipiscing.com|769|||||||
 Mckinney|Jana|Ashely Perkins|Meghan@rhoncus.com|522|||||||
Mccarthy|Gage|Kiara Solomon|Cathleen@ornare.us|462|||||||

What shell are you using?

/data/files/jlj > echo $0
-bash
/data/files/jlj > uname -a
SunOS ux562z3 5.10 Generic_150400-17 sun4u sparc SUNW,SPARC-Enterprise

That may be caused by a file in Windows format. Convert it to Unix-style first:

tr -d '\r' <dtd.dat >dtd.unix.dat
2 Likes

thank you :),