AWK - Hard to understand for me

ambarginni · March 15, 2012, 6:23am

Hi All,

I have googled and spent few hours to understand about AWK usage... But still not very much clear....

Upto my understanding 'AWK' deals with files... It reads the file takes it as a input process it and gives output or writes the output to an another file...

>> Please correct me if I am wrong OR make me more clear and precise

When I tried executing below command in the command prompt

awk' '{print$1}' SNA_DB_UPG.list

It does not take this as a command and further seeking for a terminator... :o

Further I felt better to give some file to print the output as below:

awk' '{print$1}' SNA_DB_UPG.list >> test

But Alas no luck..

Please share few more examples... I gone through with older posts as well.. But nothing clicked me..

Thanks in Advance: Ambar

itkamaraj · March 15, 2012, 6:25am

we need to give space between the print and $1

 
print $1

ambarginni · March 15, 2012, 6:29am

But still .. after giving space also .. It is not working for me..

itkamaraj · March 15, 2012, 6:31am

just noticed..

please remove the highlighted single quote

 
awk' '{print $1}' SNA_DB_UPG.list

ambarginni · March 15, 2012, 6:38am

Now it worked!! Thank you very much...

But can you please elaborate me with the logic as well.. As i am trying to understand the 'AWK'... what is all about - I know it is huge topic but the basic understanding so that I can read it further in google

Thanks again itkamaraj.. I really feel good after executing my first command using 'awk'

itkamaraj · March 15, 2012, 6:40am

you can learn more in this page ( very good for awk and sed commands )

Awk - A Tutorial and Introduction - by Bruce Barnett
Welcome to The UNIX Grymoire!

ambarginni · March 15, 2012, 11:37am

When I executed the below code in command prompt again the same problem.... Seeking for a delimiter and not taking the below as a executable code

BEGIN { print "SNA_DB_UPG.list"," }
{ print $8, "\t", $3}
END { print " - DONE -" }

It is requested to please explain me.

Scrutinizer · March 15, 2012, 11:39am

There are three double quotes on the BEGIN line

itkamaraj · March 15, 2012, 11:39am

remove the double quotes from the "SNA_DB_UP

ambarginni · March 16, 2012, 1:37am

After removing quotes .. the below code doesn't worked for me....

BEGIN { print SNA_DB_UPG.list "|" }
{ print $8, "\t", $3}
END { print " - DONE -" }

itkamaraj · March 16, 2012, 1:40am

what error you are geting ?

how you are executing ?

ambarginni · March 16, 2012, 1:43am

The output is as below: (running on command prompt)

sys,root # BEGIN { print SNA_DB_UPG.list "|" }
sh: BEGIN:  not found.
sys,root # { print $8, "\t", $3}
> END { print " - DONE -" }
>

itkamaraj · March 16, 2012, 2:10am

what you actually trying to do ?

parsing any file ?

how the file looks like ?

 
awk 'BEGIN { print SNA_DB_UPG.list "|" }{ print $8, "\t", $3}END { print " - DONE -" }' filename.txt

ambarginni · March 16, 2012, 3:02am

Thanks itkamaraj,

I have to do small change in the code (tiny project)

There is a file name SNA_DB_UPG.list which has a format:

#
#MNT4   ##R9.1.1.4
#R950   ##R9.5.0
#R9501  ##R9.5.0.1
#R960   ##R9.6.0
#############################################################
#------------MNT4-----------
MNT4|00010|msp125867|$PLATFORM_BIN/reload_PA_TABLE 1620_LM
MNT4|00020|msp127060|$PLATFORM_BIN/reload_PA_TABLE 1626_LMT
#------------R950-----------
R950|00010|msp126597|$PLATFORM_BIN/reload_PA_TABLE 1830PSS_32
R950|00020|msp126860|$PLATFORM_BIN/reload_ctl neModifier
R950|00030|msp126856|$UPG_BIN_DIR/cv950_SNAupg_001.sh
#------------R9501-----------
R9501|00010|msp127217|$PLATFORM_BIN/reload_ctl neModifier
R9501|00020|msp127540|$PLATFORM_BIN/reload_ctl layerRateMap
R9501|00030|msp127060|$PLATFORM_BIN/reload_PA_TABLE 1626_LMT
#------------R960-----------
R960|00010|msp131228|$PLATFORM_BIN/reload_ctl neModifier
R960|00020|msp131233|$UPG_BIN_DIR/cv960_NWCupg_001.sh

From here I need to read this file and create two seperate file say SPEC_INSTR.list

MNT4|00010|msp125867|$PLATFORM_BIN/reload_PA_TABLE 1620_LM
MNT4|00020|msp127060|$PLATFORM_BIN/reload_PA_TABLE 1626_LMT
R950|00010|msp126597|$PLATFORM_BIN/reload_PA_TABLE 1830PSS_32
R950|00020|msp126860|$PLATFORM_BIN/reload_ctl neModifier
R950|00030|msp126856|$UPG_BIN_DIR/cv950_SNAupg_001.sh
R9501|00010|msp127217|$PLATFORM_BIN/reload_ctl neModifier
R9501|00020|msp127540|$PLATFORM_BIN/reload_ctl layerRateMap
R9501|00030|msp127060|$PLATFORM_BIN/reload_PA_TABLE 1626_LMT
R960|00010|msp131228|$PLATFORM_BIN/reload_ctl neModifier
R960|00020|msp131233|$UPG_BIN_DIR/cv960_NWCupg_001.sh

and the other file with DISTINCT_MODEL.list

$PLATFORM_BIN/reload_PA_TABLE 1620_LM
$PLATFORM_BIN/reload_PA_TABLE 1626_LMT
$PLATFORM_BIN/reload_PA_TABLE 1830PSS_32
$PLATFORM_BIN/reload_ctl neModifier
$UPG_BIN_DIR/cv950_SNAupg_001.sh
$PLATFORM_BIN/reload_ctl neModifier (remove it - only distinct)
$PLATFORM_BIN/reload_ctl layerRateMap
$PLATFORM_BIN/reload_PA_TABLE 1626_LMT (remove it - only distinct)
$PLATFORM_BIN/reload_ctl neModifier (remove it - only distinct)
$UPG_BIN_DIR/cv960_NWCupg_001.sh

After this I need to execute the SPEC_INSTR.list first command/line i.e.

MNT4|00010|msp125867|$PLATFORM_BIN/reload_PA_TABLE 1620_LM

Only If the fourth field i.e.

$PLATFORM_BIN/reload_PA_TABLE 1620_LM

is present in the SPEC_INSTR.list file. If present after executing this command we need to delete the entry of 1620_LM from the SPEC_INSTR.list file
and further execute the same until the last command/line of the SPEC_INSTR.list file.

itkamaraj · March 16, 2012, 3:12am

 
$ nawk -F\| '!/#/{print $4}' SNA_DB_UPG.list | sort -u
$PLATFORM_BIN/reload_PA_TABLE 1620_LM
$PLATFORM_BIN/reload_PA_TABLE 1626_LMT
$PLATFORM_BIN/reload_PA_TABLE 1830PSS_32
$PLATFORM_BIN/reload_ctl layerRateMap
$PLATFORM_BIN/reload_ctl neModifier
$UPG_BIN_DIR/cv950_SNAupg_001.sh
$UPG_BIN_DIR/cv960_NWCupg_001.sh

ambarginni · March 16, 2012, 4:27am

Thank you sooo much.. itkamaraj.

It worked for me... but can you please explain me

awk -F\| '!/#/{print $4}' SNA_DB_UPG.list | sort -u

nawk doesn't seems to be working for my shell...

itkamaraj · March 16, 2012, 4:44am

-F\| --> Filed seperator is |

!/#/ --> process the lines which is not having #

print $4 --> print the 4th filed ( seperated by | )

sort -u --> print only unique values from the awk command output

ambarginni · March 16, 2012, 4:51am

Kudos to you itkamaraj!!

YOU ARE GOD OF UNIX!!

ambarginni · March 26, 2012, 6:50am

Thanks to all readers and specially to itkamaraj...

But I stucked with the below requirement please help me... I wanted the output as:

I do NOT wanted to sort the command/fourth field values/instructions
And also want to remove the duplicate values

As below:

$PLATFORM_BIN/reload_PA_TABLE 1620_LM
$PLATFORM_BIN/reload_PA_TABLE 1626_LMT
$PLATFORM_BIN/reload_PA_TABLE 1830PSS_32
$PLATFORM_BIN/reload_ctl neModifier
$UPG_BIN_DIR/cv950_SNAupg_001.sh
$PLATFORM_BIN/reload_ctl layerRateMap
$UPG_BIN_DIR/cv960_NWCupg_001.sh

Scrutinizer · March 26, 2012, 7:16am

Try:

awk -F\| '!/^#/{print;if(!A[$4]++)print $4>f}' f=DISTINCT_MODEL.list SNA_DB_UPG.list > SPEC_INSTR.list