how to parse this file and obtain a .csv or .xls

Hello Expert,
I have a file in the following format:

SYNTAX_VERSION 5
 
MONITOR "NAME_TEMPLATES"
        DESCRIPTION "Monitors for contents of error "
        INTERVAL "1m"
        MONPROG "script.sh NAME_TEMPLATES"
        MAXTHRESHOLD
        GEN_BELOW_RESET
        SEVERITY Major
        APPLICATION "script.sh"
        MSGGRP "MSG"
        MSGCONDITIONS
                DESCRIPTION "Has NEW errors"
                CONDITION_ID "00000-34dc-25e0-1cfe-0a088b060000"
                CONDITION
                        OBJECT "<#>diff<*.folder>"
                        THRESHOLD 0.500000
                        RESET 0.500000
                SET
                        OBJECT "<$OPTION(area)>"
                        TEXT "There have been new failures Processing the <$OPTION(area)> 
file on Central.
                        AUTOACTION "ls <folder>" ANNOTATE
                        HELPTEXT "OPS
=========
Please call support team"
                        HELP "0d152f3c-59dc-71e0-1cfe-0a034b060000"
                DESCRIPTION "Has ANY errors"
                CONDITION_ID "d26dda06-55f2-71e0-0025-0a034b060000"
                CONDITION
                        OBJECT "<*.folder>"
                        THRESHOLD 0.500000
                        RESET 0.500000
                SET
                        OBJECT "<$OPTION(area)>"
                        TEXT "There has been a failure Processing the <$OPTION(area)> file 
                        HELPTEXT "OPS
=========
Please call out the support team"
                       others instruction
 others instruction
 HELP "0d152f3c-59dc-71e0-1cfe-0a034b060000"

I would like to transform it in .csv or .xls file.

I did the following command
1)

<more filename>| awk -F\" '{print $1","$2}'> /tmp/monitor.csv

2) But I need to show in two field also the rows without double quote:

                        THRESHOLD 0.500000
                        RESET 0.500000

3) Further (and this is more complicate to achieve) I need the part of this file after HELPTEXT and before HELP will be visualized in a unique row:

                        HELPTEXT "OPS
=========
Please call out the support team"
                       others instruction
 others instruction
 HELP "0d152f3c-59dc-71e0-1cfe-0a034b060000"

Is possible to achieve all these 3 steps with awk and in a unique script? :confused:

Thanks in advance for any help,
Ant-one

Could this help you?

 perl -nle 'print $_ if /\bTHRESHOLD\b/../\bRESET\b/; print $_ if /\bHELPTEXT\b/../\bHELP\b/'  inputfile

Hello,
when I do your command, I receive the follow result:

                        THRESHOLD 0.500000
                        RESET 0.500000

My desired output is in .csv:

MONITOR,NAME_TEMPLATES
        DESCRIPTION,Monitors for contents of error
        INTERVAL,1m
        MONPROG ,script.sh NAME_TEMPLATES,
... 
..
..
..
..
 (same row 2 fields)THRESHOLD, 0.500000
                        RESET, 0.500000
                        HELPTEXT ,OPS,
 (same row)         ========= Please call out the support team" others instruction others instruction

Thanks,
Ant-one

Hi Ant-one, Try this,

awk -F\" '{gsub(/^ +/,"");if(/HELPTEXT/) {printf $0;flg=1;next}if(flg==1 && !/HELP/){printf $0" ";next} if(/HELP/){flg=0;print;next}if(/THRESHOLD / || /RESET /){gsub("THRESHOLD ","THRESHOLD,");gsub("RESET ","RESET,");print;next}print $1","$2}' inputfile

Hi,
I received the following error:

Thanks,
Ant-one

---------- Post updated at 08:21 AM ---------- Previous update was at 07:44 AM ----------

Hi,
first of all many thanks.
It worked fine for the major part, this is the ouput of your command:

Is it possible to split SEVERITY Major in 2 fields, please?

The text shoul be in 2 part:

TEXT ,There have been new failures Processing the <$OPTION(area)> file on Central.

Instead:

And the last part

Should be without the part HELP "0d152f3c-59dc-71e0-1cfe-0a034b060000" as follow:

Thanks in advance for any suggestion.

Ant-one

Also try

nawk '/HELPTEXT/{x=$0;getline;do {x=x" "$0 ; getline}while ($0!~/HELP /);sub(/^ */,"",x);sub(" ",", ",x);sub(/  */," ",x);print x}NF>1{sub($1,$1",",$0)}1' infile

or

 nawk '/HELPTEXT/{x=$0;getline;do{x=x" "$0;getline}while($0!~/HELP /);sub(/ ["]OPS/,", \"OPS,",x);print x}NF>1{sub($1,$1",",$0)}1' infile

Hi,
thanks for your suggestion but there is not nawk on my server.

I have added severity into pravin27 code:

Regards,
Ant-one

Just make a try with awk instead of nawk (the syntax is similar).

sed 's/^  *//;s/  */ /g;s/  *$//' infile | awk '/HELPTEXT/{x=$0;getline;do{x=x" "$0;getline}while($0!~/HELP /);sub(/ ["]OPS/,", \"OPS,",x);print x}NF>1{sub($1,$1",",$0)}1'

---------- Post updated at 04:04 PM ---------- Previous update was at 04:02 PM ----------

$ sed 's/^  *//;s/  */ /g;s/  *$//' tst | nawk '/HELPTEXT/{x=$0;getline;do{x=x" "$0;getline}while($0!~/HELP /);sub(/ ["]OPS/,", \"OPS,",x);print x}NF>1{sub($1,$1",",$0)}1'
SYNTAX_VERSION, 5

MONITOR, "NAME_TEMPLATES"
DESCRIPTION, "Monitors for contents of error "
INTERVAL, "1m"
MONPROG, "script.sh NAME_TEMPLATES"
MAXTHRESHOLD
GEN_BELOW_RESET
SEVERITY, Major
APPLICATION, "script.sh"
MSGGRP, "MSG"
MSGCONDITIONS
DESCRIPTION, "Has NEW errors"
CONDITION_ID, "00000-34dc-25e0-1cfe-0a088b060000"
CONDITION
OBJECT, "<#>diff<*.folder>"
THRESHOLD, 0.500000
RESET, 0.500000
SET
OBJECT, "<$OPTION(area)>"
TEXT, "There have been new failures Processing the <$OPTION(area)>
file, on Central.
AUTOACTION, "ls <folder>" ANNOTATE
HELPTEXT, "OPS, ========= Please call support team"
HELP, "0d152f3c-59dc-71e0-1cfe-0a034b060000"
DESCRIPTION, "Has ANY errors"
CONDITION_ID, "d26dda06-55f2-71e0-0025-0a034b060000"
CONDITION
OBJECT, "<*.folder>"
THRESHOLD, 0.500000
RESET, 0.500000
SET
OBJECT, "<$OPTION(area)>"
TEXT, "There has been a failure Processing the <$OPTION(area)> file
HELPTEXT, "OPS, ========= Please call out the support team" others instruction others instruction
HELP, "0d152f3c-59dc-71e0-1cfe-0a034b060000"
$

Thanks your code does what I want!

Only there is a little issue, the prompt remain in hung at the end of the execution of your command, I need to stop it with CTRL+C (not good to use it in automatic script as it is my intention).

Any help?

Regards,

Ant-one

Please make sure you did no typo error (don't forget any simple nor double quote).

1) Which OS/shell do you run ?

2) Please copy/paste the exact command you ran ?

Hi,
1) /usr/bin/sh HP-UX 11.31

2)

Regards,
Ant-one

(I ran it both on a linux & solaris and both worked fine and i had no such issue)

# sed 's/^  *//;s/  */ /g;s/  *$//' yourfile | awk '/HELPTEXT/{x=$0;getline;while($0!~/HELP /){x=x" "$0;getline};sub(/ ["]OPS/,", \"OPS,",x);print x}NF>1{sub($1,$1",",$0)}1'
SYNTAX_VERSION, 5

MONITOR, "NAME_TEMPLATES"
DESCRIPTION, "Monitors for contents of error "
INTERVAL, "1m"
MONPROG, "script.sh NAME_TEMPLATES"
MAXTHRESHOLD
GEN_BELOW_RESET
SEVERITY, Major
APPLICATION, "script.sh"
MSGGRP, "MSG"
MSGCONDITIONS
DESCRIPTION, "Has NEW errors"
CONDITION_ID, "00000-34dc-25e0-1cfe-0a088b060000"
CONDITION
OBJECT, "<#>diff<*.folder>"
THRESHOLD, 0.500000
RESET, 0.500000
SET
OBJECT, "<$OPTION(area)>"
TEXT, "There have been new failures Processing the <$OPTION(area)>
file, on Central.
AUTOACTION, "ls <folder>" ANNOTATE
HELPTEXT, "OPS, ========= Please call support team"
HELP, "0d152f3c-59dc-71e0-1cfe-0a034b060000"
DESCRIPTION, "Has ANY errors"
CONDITION_ID, "d26dda06-55f2-71e0-0025-0a034b060000"
CONDITION
OBJECT, "<*.folder>"
THRESHOLD, 0.500000
RESET, 0.500000
SET
OBJECT, "<$OPTION(area)>"
TEXT, "There has been a failure Processing the <$OPTION(area)> file
HELPTEXT, "OPS, ========= Please call out the support team" others instruction others instruction
HELP, "0d152f3c-59dc-71e0-1cfe-0a034b060000"

---------- Post updated at 04:44 PM ---------- Previous update was at 04:41 PM ----------

Watch out for the number of space in the sed statement !

---------- Post updated at 04:47 PM ---------- Previous update was at 04:44 PM ----------

sed 's/^  *//;s/  */ /g;s/  *$//' monitor

The wildcard is preceeded by 2 space every time
<space><space>*

(if it is not the case, it is an error : you should fix it )

hello,
i have tried split the command in two part:

-when I use the sed statement only without the awk it works fine and I get back the prompt.

hen I use the awk part it hungs:

Instead if I use this command

I get the following output without hung:

:wall:

could you show me what gives :

head myfile2

??

As stated in my previous post : your sed statement is WRONG :

You must put 2 space before every * in the sed statement (i put them in [ ] so you can see where they are and how many of them should be ):

sed 's/^[ ][ ]*//;s/[ ][ ]*$//;s/[ ][ ]/[ ]/g' infile

---------- Post updated at 06:07 PM ---------- Previous update was at 05:54 PM ----------

Look the difference if one space is missing, the output is messed up :

# sed 's/^ *//;s/ */ /g;s/ *$//' tst | head -3
 S Y N T A X _ V E R S I O N 5

 M O N I T O R " N A M E _ T E M P L A T E S "
# sed 's/^ *//;s/  */ /g;s/ *$//' tst | head -3
SYNTAX_VERSION 5

MONITOR "NAME_TEMPLATES"
#

But to make the code more secure, this missing space should also be added before the other * :

# sed 's/^  *//;s/  */ /g;s/  *$//' tst | head -3
SYNTAX_VERSION 5

MONITOR "NAME_TEMPLATES"

Hello,
sorry for the later answer.
The sed statement is fine (I have checked 2 spaces a lot of time) and it does not hung:

But when I use the awk part it keep hung.
What a shame!

Thanks,
Ant-one

just try to copy/paste this :

sed 's/^  *//;s/  */ /g;s/  *$//' monitor | awk '/HELPTEXT/{x=$0;getline;do{x=x" "$0;getline}while($0!~/HELP /);sub(/ ["]OPS/,", \"OPS,",x);print x}NF>1{sub($1,$1",",$0)}1'

Same bad result, it hungs!

Thanks,
Ant-one

I am unable to reproduce your problem. I have tested my code on solaris (nawk) and linux (awk) both work fine, i have no idea why your HPUX system is not able to handle it correctly.

By the way, i noticed something strange : when you copy past in your previous post #15, i still see only one space before the '*' , even when i copy past what you have posted, it looks as if you were missing the second required space, whereas you told me you checked you have 2 of them, i don't understand why one seems have been unexpectedly eaten.

Could you please upload your "monitor" file (if it is not formatted the same way than the example you gave, then, maybe some /HELP / occurrence are missing and the while loop then turns into an endless loop ?