Padding a csv value with 0's

Shell Programming and Scripting
1

I have this csv file that I would like to sort on the 20th and 21st field. They are high lighted below. My challenge is that when I sort on those fields they are not in order as I would have liked. It seems like I have to pad those fields to the longest value in that fields data.

OrderNum,MerrillRecipientID,CustomerClass,MerrillItemNum,PODTemplateID,GridCode,AetnaDocID,MemberID,FirstName,MI,LastName,Address1,Address2,Address3,City,State,Zip,Country,OEL,PalletNum,PckgNum,IMBCode,ProcDate
"M394993","M39499300010000001","0GH","3GH000503","PDP","BO","1011250","MEBB04CB","Name","","Name","address","","","City","SC","29170-2043","","*******AUTO**SCH 5-DIGIT 29033","P1","PK5","2031100094470495539729170204309","3GH000503","August 26, 2013"
"M394993","M39499300010000002","0GH","3GH000503","PDP","BO","1011572","MEBB07GB","Name","G","Name","address","","","City","SC","29020-2912","","*********AUTO**SCH 3-DIGIT 290","P1","PK1","3031100094470495580529020291210","3GH000503","August 26, 2013"
"M394993","M39499300010000003","0GH","3GH000503","PDP","BO","1011693","MEBB08MP","Name","B","Name","address","","","City","SC","29061-9447","","*********AUTO**SCH 3-DIGIT 290","P1","PK2","3031100094470495583729061944757","3GH000503","August 26, 2013"
"M394993","M39499300010000004","0GH","3GH000503","PDP","BO","1011751","MEBB097M","Name","A","Name","address","","","City","SC","29645-0433","","*************AUTO**3-DIGIT 296","P1","PK31","3031100094470495629629645043333","3GH000503","August 26, 2013"
"M394993","M39499300010000005","0GH","3GH000503","PDP","BO","1012075","MEBB0K4L","Name","E","Name","address","","","City","SC","29682-9634","","*************AUTO**3-DIGIT 296","P1","PK33","3031100094470495637929682963428","3GH000503","August 26, 2013"
"M394993","M39499300010000006","0GH","3GH000503","PDP","BO","1012437","MEBB0TWQ","Name","R","Name","address","","","City","SC","29505-3030","","*******AUTO**SCH 5-DIGIT 29501","P1","PK24","2031100094470495556429505303050","3GH000503","August 26, 2013"
"M394993","M39499300010000007","0GH","3GH000503","PDP","BO","1012750","MEBB0YJY","Name","L","Name","address","","","City","SC","29642-3006","","***********AUTO**5-DIGIT 29642","P1","PK38","2031100094470495567529642300601","3GH000503","August 26, 2013"

So from the above data I need to have the file look like so

OrderNum,MerrillRecipientID,CustomerClass,MerrillItemNum,PODTemplateID,GridCode,AetnaDocID,MemberID,FirstName,MI,LastName,Address1,Address2,Address3,City,State,Zip,Country,OEL,PalletNum,PckgNum,IMBCode,ProcDate
"M394993","M39499300010000001","0GH","3GH000503","PDP","BO","1011250","MEBB04CB","Name","","Name","address","","","City","SC","29170-2043","","*******AUTO**SCH 5-DIGIT 29033","P1","PK05","2031100094470495539729170204309","3GH000503","August 26, 2013"
"M394993","M39499300010000002","0GH","3GH000503","PDP","BO","1011572","MEBB07GB","Name","G","Name","address","","","City","SC","29020-2912","","*********AUTO**SCH 3-DIGIT 290","P1","PK01","3031100094470495580529020291210","3GH000503","August 26, 2013"
"M394993","M39499300010000003","0GH","3GH000503","PDP","BO","1011693","MEBB08MP","Name","B","Name","address","","","City","SC","29061-9447","","*********AUTO**SCH 3-DIGIT 290","P1","PK02","3031100094470495583729061944757","3GH000503","August 26, 2013"
"M394993","M39499300010000004","0GH","3GH000503","PDP","BO","1011751","MEBB097M","Name","A","Name","address","","","City","SC","29645-0433","","*************AUTO**3-DIGIT 296","P1","PK31","3031100094470495629629645043333","3GH000503","August 26, 2013"
"M394993","M39499300010000005","0GH","3GH000503","PDP","BO","1012075","MEBB0K4L","Name","E","Name","address","","","City","SC","29682-9634","","*************AUTO**3-DIGIT 296","P1","PK33","3031100094470495637929682963428","3GH000503","August 26, 2013"
"M394993","M39499300010000006","0GH","3GH000503","PDP","BO","1012437","MEBB0TWQ","Name","R","Name","address","","","City","SC","29505-3030","","*******AUTO**SCH 5-DIGIT 29501","P1","PK24","2031100094470495556429505303050","3GH000503","August 26, 2013"
"M394993","M39499300010000007","0GH","3GH000503","PDP","BO","1012750","MEBB0YJY","Name","L","Name","address","","","City","SC","29642-3006","","***********AUTO**5-DIGIT 29642","P1","PK38","2031100094470495567529642300601","3GH000503","August 26, 2013"

the P1 field could be P100 and so I would need to pad P1 to P001. But really it justs need to be whatever the max length is. I can sort the file on the two fields but not sure how to pad them.

Thanks in advance for your help.

2 
 
bash-3.2$ cat txt
OrderNum,MerrillRecipientID,CustomerClass,MerrillItemNum,PODTemplateID,GridCode,AetnaDocID,MemberID,FirstName,MI,LastName,Address1,Address2,Address3,City,State,Zip,Country,OEL,PalletNum,PckgNum,IMBCode,ProcDate
"M394993","M39499300010000001","0GH","3GH000503","PDP","BO","1011250","MEBB04CB","Name","","Name","address","","","City","SC","29170-2043","","*******AUTO**SCH 5-DIGIT 29033","P1","PK5","2031100094470495539729170204309","3GH000503","August 26, 2013"
"M394993","M39499300010000002","0GH","3GH000503","PDP","BO","1011572","MEBB07GB","Name","G","Name","address","","","City","SC","29020-2912","","*********AUTO**SCH 3-DIGIT 290","P1","PK1","3031100094470495580529020291210","3GH000503","August 26, 2013"
"M394993","M39499300010000003","0GH","3GH000503","PDP","BO","1011693","MEBB08MP","Name","B","Name","address","","","City","SC","29061-9447","","*********AUTO**SCH 3-DIGIT 290","P1","PK2","3031100094470495583729061944757","3GH000503","August 26, 2013"
"M394993","M39499300010000004","0GH","3GH000503","PDP","BO","1011751","MEBB097M","Name","A","Name","address","","","City","SC","29645-0433","","*************AUTO**3-DIGIT 296","P1","PK31","3031100094470495629629645043333","3GH000503","August 26, 2013"
"M394993","M39499300010000005","0GH","3GH000503","PDP","BO","1012075","MEBB0K4L","Name","E","Name","address","","","City","SC","29682-9634","","*************AUTO**3-DIGIT 296","P1","PK33","3031100094470495637929682963428","3GH000503","August 26, 2013"
"M394993","M39499300010000006","0GH","3GH000503","PDP","BO","1012437","MEBB0TWQ","Name","R","Name","address","","","City","SC","29505-3030","","*******AUTO**SCH 5-DIGIT 29501","P1","PK24","2031100094470495556429505303050","3GH000503","August 26, 2013"
"M394993","M39499300010000007","0GH","3GH000503","PDP","BO","1012750","MEBB0YJY","Name","L","Name","address","","","City","SC","29642-3006","","***********AUTO**5-DIGIT 29642","P1","PK38","2031100094470495567529642300601","3GH000503","August 26, 2013"
bash-3.2$ 
bash-3.2$ 
bash-3.2$ 
bash-3.2$ awk -F, '{print $20$21"="$0}' txt | perl -MSort::Naturally -lne ' my($key,$val)=split(/=/,$_); if($key eq 'PalletNumPckgNum'){print $val; next;} $x{$key}=$val; END { foreach (nsort keys %x ){ print $x{$_}; } }' 
OrderNum,MerrillRecipientID,CustomerClass,MerrillItemNum,PODTemplateID,GridCode,AetnaDocID,MemberID,FirstName,MI,LastName,Address1,Address2,Address3,City,State,Zip,Country,OEL,PalletNum,PckgNum,IMBCode,ProcDate
"M394993","M39499300010000002","0GH","3GH000503","PDP","BO","1011572","MEBB07GB","Name","G","Name","address","","","City","SC","29020-2912","","*********AUTO**SCH 3-DIGIT 290","P1","PK1","3031100094470495580529020291210","3GH000503","August 26, 2013"
"M394993","M39499300010000003","0GH","3GH000503","PDP","BO","1011693","MEBB08MP","Name","B","Name","address","","","City","SC","29061-9447","","*********AUTO**SCH 3-DIGIT 290","P1","PK2","3031100094470495583729061944757","3GH000503","August 26, 2013"
"M394993","M39499300010000001","0GH","3GH000503","PDP","BO","1011250","MEBB04CB","Name","","Name","address","","","City","SC","29170-2043","","*******AUTO**SCH 5-DIGIT 29033","P1","PK5","2031100094470495539729170204309","3GH000503","August 26, 2013"
"M394993","M39499300010000006","0GH","3GH000503","PDP","BO","1012437","MEBB0TWQ","Name","R","Name","address","","","City","SC","29505-3030","","*******AUTO**SCH 5-DIGIT 29501","P1","PK24","2031100094470495556429505303050","3GH000503","August 26, 2013"
"M394993","M39499300010000004","0GH","3GH000503","PDP","BO","1011751","MEBB097M","Name","A","Name","address","","","City","SC","29645-0433","","*************AUTO**3-DIGIT 296","P1","PK31","3031100094470495629629645043333","3GH000503","August 26, 2013"
"M394993","M39499300010000005","0GH","3GH000503","PDP","BO","1012075","MEBB0K4L","Name","E","Name","address","","","City","SC","29682-9634","","*************AUTO**3-DIGIT 296","P1","PK33","3031100094470495637929682963428","3GH000503","August 26, 2013"
"M394993","M39499300010000007","0GH","3GH000503","PDP","BO","1012750","MEBB0YJY","Name","L","Name","address","","","City","SC","29642-3006","","***********AUTO**5-DIGIT 29642","P1","PK38","2031100094470495567529642300601","3GH000503","August 26, 2013"

You need this perl module Sort::Naturally
Sort::Naturally - search.cpan.org

Just make a directory called Sort, and put Naturally.pm within the Sort directory.

Save the csv file up one level of the Sort directory and launch that command.

3

You may try to develop on

sort -t, -nk 21.4 file
4

Not having tried this yet, But can I sort taking both 20 and 21 into consideration? I will be able to mess with this in about 1/2 hour or so.

Mr.Bean, I will try your method if I can't get anything else to work. I would prefer to no have to install anything to make this happen.

Thanks to both on the reply.

5

How do you want field 21 sorted...lexically or numerically...and post a sample of how the output looks like because from your post there seems to be no difference in the input and output...and you can try the sort below which will divvy up field 21 into 2 parts i.e. the first 3 characters are sorted lexically and the rest are sorted numerically...is this something you are looking for...

sort -t, -k21.1,21.3 -k21.4,21n file
6

This solution is more generic.
Create new keyfield(s) which are usable for sort.
Sort it and then remove those extra fields. This not works if fld value include comma (,).

awk 'BEGIN { FS=","
             OFS=","
           }
NR==1 { print $0,"A","0","AA","0" ; next } # sort key for headerline
{
    f20=$20
    f21=$21
    gsub(/"/,"",f20)
    gsub(/"/,"",f21)
    f20a=substr(f20,1,1) # 1st char
    f20b=substr(f20,2)  # rest is numeric
    f21a=substr(f21,1,2) # 2 char
    f21b=substr(f21,3)   # rest is numeric
    print $0,f20a,f20b,f21a,f21b
}

' inputfile | \
        sort -t, -k 26,26 -nk 27,27 -k 28,28 -nk 29,29 | \
        awk 'BEGIN { FS=","  }
                {
                  for (i=1;i<=24;i++)
                        printf("%s,",$i)
                  printf("%s\n",$25)
                } '
7

I ended up with a file that looked like so

OrderNum,MerrillRecipientID,CustomerClass,MerrillItemNum,PODTemplateID,GridCode,AetnaDocID,MemberID,FirstName,MI,LastName,Address1,Address2,Address3,City,State,Zip,Country,OEL,PalletNum,PckgNum,IMBCode,LocationID,ProcDate
,A
"V670443","V67044300010000320","0GH","3GH000501","HMO_MAPD","2E","1001086","ME091036","Name","G","Name","Address SPRUCE ST APT 1710","","","PHILADELPHIA","PA","19102-4554","","***********AUTO**5-DIGIT 19102","P1","PK1","2030110676000685569319102455435","3GH000501","August 28, 2013"
"V670443","V67044300010000698","0GH","3GH000501","HMO_MAPD","2E","1003663","ME321579","Name","N","Name","address SPRUCE ST APT 709","THE DRAKE","","PHILADELPHIA","PA","19102-4551","","***********AUTO**5-DIGIT 19102","P1","PK1","2030110676000685569419102455184","3GH000501","August 28, 2013"
"V670443","V67044300010001103","0GH","3GH000501","HMO_MAPD","2E","1038732","MEBBC41B","Name","J","Name","address LOCUST ST APT 20N","","","PHILADELPHIA","PA","19102-4212","","***********AUTO**5-DIGIT 19102","P1","PK1","2030110676000685569519102421224","3GH000501","August 28, 2013"
"V670443","V67044300010001179","0GH","3GH000501","HMO_MAPD","2E","1039945","MEBBFYZM","Name","J","Name","address S HICKS ST APT 3A","","","PHILA","PA","19102-4947","","***********AUTO**5-DIGIT 19102","P1","PK1","2030110676000685569619102494731","3GH000501","August 28, 2013"
"V670443","V67044300010001825","0GH","3GH000501","HMO_MAPD","2E","1053739","MEBBQNQL","Name","D","Name","address LOCUST ST APT 16N","","","PHILA","PA","19102-4210","","***********AUTO**5-DIGIT 19102","P1","PK1","2030110676000685569719102421084","3GH000501","August 28, 2013"
"V670443","V67044300010002280","0GH","3GH000501","HMO_MAPD","2E","1108001","MEBCHSQS","Name","H","Name","address LOCUST ST APT 8H","","","PHILADELPHIA","PA","19102-4206","","***********AUTO**5-DIGIT 19102","P1","PK1","2030110676000685569819102420688","3GH000501","August 28, 2013"

I will look to see why it kept that ,A line.

Thanks!