To search the pattern on the basis of date and exit code

ajju · August 27, 2014, 11:37am

Hi,

I am getting scheduler log file on daily basis from windows box which contains job status and corresponding date, date is in windows format.

I wanted to write one script which will search the pattern (Exit code) for the today's date and if code is Zero then Job Success message should be written in log else failure..

Log message is as below

"abcd.job" (abcd..bat)
	Started 8/27/2014 5:00:00 AM
"abcd.job" (abcd.bat)
	Finished 8/27/2014 5:07:00 AM
	Result: The task completed with an exit code of (0).
"PBC Analysis.job" (PBC Analysis.bat)
	Started 8/27/2014 7:00:00 AM
"Eventlog.job" (Event_clear.vbs)
	Started 8/27/2014 7:00:02 AM
"Eventlog.job" (Event_clear.vbs)
	Finished 8/27/2014 7:00:05 AM
	Result: The task completed with an exit code of (0).
"BUS Analysis.job" (BUS Analysis.bat)
	Finished 8/27/2014 7:00:10 AM
	Result: The task completed with an exit code of (0).
[ ***** Most recent entry is above this line ***** ]

So in the above log script will look only for the abcd.job and its result for the current date, other jobs should be ignore.

MY server is AIX, please assist.

rbatte1 · August 27, 2014, 12:32pm

Hello ajju,

I have a few to questions pose in response first:-

What have you tried so far?
What output/errors do you get?
What OS version are you using?
What are your preferred tools? (C, shell, perl, awk, etc.)
What logical process have you considered? (to help steer us to follow what you are trying to achieve)
Are the sections without a Finished and Result record to be considered as failures or are they considered as running?
Is the format always the same?

Most importantly, What have you tried so far?

There are probably many ways to achieve most tasks, so giving us an idea of your style and thoughts will help us guide you to an answer most suitable to you so you can adjust it to suit your needs in future.

We're all here to learn and getting the relevant information will help us all.

From

I would say that this is US format, but then that may not be correct either. Importantly, it appears that it is m/d/yyyy h:mi:ss AP so there is still a question of if there is a leading zero on the day. It would affect the options for the date command to work out the search string. Have a look at the manual page for datefor the options such as %d to see why it is important.

Have a look at the manual page for grep. As you are AIX, the -p flag may be of interest.

You will need to check what time of day you will be running this too, else you may actually need to look for the previous day's records and then you have to consider a calculation which AIX may not have built in to the date command in your version.

Regards,
Robin

Corona688 · August 27, 2014, 12:45pm

$ sed 's/^"/\n"/' input | awk -v RS="" -v FS="\n" -v D="$(date +/%m/%d/%Y)" -v JOB="abcd.job" 'BEGIN { gsub("/0", "/", D); D=substr(D,2); } ; index($1, "\""JOB"\"") && ($2 ~ D)'

"abcd.job" (abcd..bat)
        Started 8/27/2014 5:00:00 AM
"abcd.job" (abcd.bat)
        Finished 8/27/2014 5:07:00 AM
        Result: The task completed with an exit code of (0).

$

wisecracker · August 27, 2014, 1:04pm

Hi Corona688...

Third line:-

"abcd.job" (abcd..bat)

Is abcd..bat a valid Windows format?

Just curious...

Corona688 · August 27, 2014, 1:12pm

Why not? . is a valid character in a filename.

ajju · August 27, 2014, 1:49pm

thank you very much Coron

But I tried the given code its giving a NIL output though the command is running successfully.

sed 's/^"/\n"/' np_log.txt | awk -v RS="" -v FS="\n" -v D="$(date +/%m/%d/%Y)" -v JOB="abcd.job" 'BEGIN { gsub("/0", "/", D); D=substr(D,2); } ; index($1, "\""JOB"\"") && ($2 ~ D)'

No output is coming...

Np_log.txt containing the below pattern...

"CRF Analysis.job" (CRF Analysis.bat)
        Finished 8/26/2014 7:01:49 AM
        Result: The task completed with an exit code of (0).
"PULSS22.job" (PULLS22.bat)
        Started 8/26/2014 8:30:03 PM
"rtmas22.job" (rtmas22.bat)
        Finished 8/26/2014 8:33:40 PM
        Result: The task completed with an exit code of (0).
"abcd.job" (abcd.bat)
        Started /28/2014 5:00:02 AM
"abcd.job" (abcd.bat)
        Finished 8/28/2014 5:07:17 AM
        Result: The task completed with an exit code of (0).
"VRL Analysis.job" (VRL Analysis.bat)
        Started 8/27/2014 7:00:12 AM
"Eventlog.job" (Event_clear.vbs)
        Started 8/28/2014 7:00:17 AM
"Eventlog.job" (Event_clear.vbs)
        Finished 08/28/2014 7:00:19 AM
        Result: The task completed with an exit code of (1).
"VMR Analysis.job" (VMR Analysis.bat)
        Finished 8/28/2014 7:01:49 AM
        Result: The task completed with an exit code of (0).
"SPULL.job" (SPULL.bat)
         Started 8/28/2014 8:30:03 PM
"SPULL22.job" (SPULL22.bat)
        Finished 8/28/2014 8:33:40 PM
        Result: The task completed with an exit code of (0).
"WEC Analysis.job" (WEC Analysis.bat)
        Finished 8/28/2014 7:00:18 AM
        Result: The task completed with an exit code of (0).
"rtpbc22.job" (rtpbc22.bat) 8/28/2014 8:30:08 PM

Expected thing is just wanted to know what is the status of Job abcd.bat , if exit code is 0 then it shold through success message , else failure..

RudiC · August 27, 2014, 2:46pm

No surprise, given the sample file you provided. Corona688's proposal has two conditions joined by the AND operator: index($1, "\""JOB"\"") && ($2 ~ D)' (JOB holding "abcd.job"), of which the first is fulfilled by records 4 & 5, the second by record 6. No record satisfied both conditions, so no record is printed.

Corona688 · August 27, 2014, 6:27pm

Today's the 27th, the record you want is on the 28th, so it doesn't match.

ajju · August 27, 2014, 11:28pm

Hi Rudie/Corona,

Even if iam searching the for today's log no output is coming even though for the today's date job got success.

So the main purpose of this job to check the status of the job, as below

"abcd.job" (abcd.bat)
        Finished 8/28/2014 5:07:17 AM
        Result: The task completed with an exit code of (0).

If it success then it should give message like job successfully executes else job got fail.

RudiC · August 28, 2014, 4:06am

Surprise - it's working today (28.Aug.2014)!

sed 's/^"/\n"/' file3 | awk -v RS="" -v FS="\n" -v D="$(date +/%m/%d/%Y)" -v JOB="abcd.job" 'BEGIN { gsub("/0", "/", D); D=substr(D,2); } ; index($1, "\""JOB"\"") && ($2 ~ D)'
"abcd.job" (abcd.bat)
        Finished 8/28/2014 5:07:17 AM
        Result: The task completed with an exit code of (0).

ajju · August 28, 2014, 2:07pm

Hi Corona,

Sorry to say but still code doesn't work as expected.

cant we have a other way by greping multiple pattern like date, jobname, and status code , if the abcd.job for the current date have the status as

Result: The task completed with an exit code of (0)

then it will log the success in a new log else job got failed.

pasting here complete log file...

"abcd.job" (abcd.bat)
        Started 8/24/2014 5:00:00 AM
"abcd.job" (abcd.bat)
        Finished 8/24/2014 5:07:04 AM
        Result: The task completed with an exit code of (0).
"CRF Analysis.job" (CRF Analysis.bat)
        Started 8/24/2014 7:00:00 AM
"Eventlog.job" (Event_clear.vbs)
        Started 8/24/2014 7:00:01 AM
"Eventlog.job" (Event_clear.vbs)
        Finished 8/24/2014 7:00:06 AM
        Result: The task completed with an exit code of (0).
"CRF Analysis.job" (CRF Analysis.bat)
        Finished 8/24/2014 7:00:09 AM
        Result: The task completed with an exit code of (0).
"abcd.job" (abcd.bat)
        Started 8/25/2014 5:00:00 AM
"abcd.job" (abcd.bat)
        Finished 8/25/2014 5:06:58 AM
        Result: The task completed with an exit code of (0).
"CRF Analysis.job" (CRF Analysis.bat)
        Started 8/25/2014 7:00:00 AM
"CRF Analysis.job" (CRF Analysis.bat)
        Finished 8/25/2014 7:00:08 AM
        Result: The task completed with an exit code of (0).
"abcd.job" (abcd.bat)
        Started 8/26/2014 5:00:00 AM
"abcd.job" (abcd.bat)
        Finished 8/26/2014 5:07:01 AM
        Result: The task completed with an exit code of (0).
"CRF Analysis.job" (CRF Analysis.bat)
        Started 8/26/2014 7:00:00 AM
"CRF Analysis.job" (CRF Analysis.bat)
        Finished 8/26/2014 7:00:08 AM
        Result: The task completed with an exit code of (0).
"abcd.job" (abcd.bat)
        Started 8/27/2014 5:00:00 AM
"abcd.job" (abcd.bat)
        Finished 8/27/2014 5:07:00 AM
        Result: The task completed with an exit code of (0).
"CRF Analysis.job" (CRF Analysis.bat)
        Started 8/27/2014 7:00:00 AM
"Eventlog.job" (Event_clear.vbs)
        Started 8/27/2014 7:00:02 AM
"Eventlog.job" (Event_clear.vbs)
        Finished 8/27/2014 7:00:05 AM
        Result: The task completed with an exit code of (0).
"CRF Analysis.job" (CRF Analysis.bat)
        Finished 8/27/2014 7:00:10 AM
        Result: The task completed with an exit code of (0).
"abcd.job" (abcd.bat)
        Started 8/28/2014 5:00:00 AM
"abcd.job" (abcd.bat)
        Finished 8/28/2014 5:07:05 AM
        Result: The task completed with an exit code of (0).
"abcd.job" (abcd.bat)
        Started 8/29/2014 5:00:00 AM
"abcd.job" (abcd.bat)
        Finished 8/29/2014 5:07:05 AM
        Result: The task completed with an exit code of (0).
"CRF Analysis.job" (CRF Analysis.bat)
        Started 8/28/2014 7:00:00 AM
"CRF Analysis.job" (CRF Analysis.bat)
        Finished 8/28/2014 7:00:09 AM
        Result: The task completed with an exit code of (0).
[ ***** Most recent entry is above this line ***** ]
"abcd.job" (abcd.bat)
        Started 8/24/2014 5:00:00 AM
"abcd.job" (abcd.bat)
        Finished 8/24/2014 5:07:04 AM
        Result: The task completed with an exit code of (0).
"CRF Analysis.job" (CRF Analysis.bat)
        Started 8/24/2014 7:00:00 AM
"Eventlog.job" (Event_clear.vbs)
        Started 8/24/2014 7:00:01 AM
"Eventlog.job" (Event_clear.vbs)
        Finished 8/24/2014 7:00:06 AM
        Result: The task completed with an exit code of (0).
"CRF Analysis.job" (CRF Analysis.bat)
        Finished 8/24/2014 7:00:09 AM
        Result: The task completed with an exit code of (0).
"abcd.job" (abcd.bat)
        Started 8/25/2014 5:00:00 AM
"abcd.job" (abcd.bat)
        Finished 8/25/2014 5:06:58 AM
        Result: The task completed with an exit code of (0).
"CRF Analysis.job" (CRF Analysis.bat)
        Started 8/25/2014 7:00:00 AM
"CRF Analysis.job" (CRF Analysis.bat)
        Finished 8/25/2014 7:00:08 AM
        Result: The task completed with an exit code of (0).
"abcd.job" (abcd.bat)
        Started 8/26/2014 5:00:00 AM
"abcd.job" (abcd.bat)
        Finished 8/26/2014 5:07:01 AM
        Result: The task completed with an exit code of (0).
"CRF Analysis.job" (CRF Analysis.bat)
        Started 8/26/2014 7:00:00 AM
"CRF Analysis.job" (CRF Analysis.bat)
        Finished 8/26/2014 7:00:08 AM
        Result: The task completed with an exit code of (0).
"abcd.job" (abcd.bat)
        Started 8/27/2014 5:00:00 AM
"abcd.job" (abcd.bat)
        Finished 8/27/2014 5:07:00 AM
        Result: The task completed with an exit code of (0).
"CRF Analysis.job" (CRF Analysis.bat)
        Started 8/27/2014 7:00:00 AM
"Eventlog.job" (Event_clear.vbs)
        Started 8/27/2014 7:00:02 AM
"Eventlog.job" (Event_clear.vbs)
        Finished 8/27/2014 7:00:05 AM
        Result: The task completed with an exit code of (0).
"CRF Analysis.job" (CRF Analysis.bat)
        Finished 8/27/2014 7:00:10 AM
        Result: The task completed with an exit code of (0).
"abcd.job" (abcd.bat)
        Started 8/29/2014 5:00:00 AM
"abcd.job" (abcd.bat)
        Finished 8/29/2014 5:07:05 AM
        Result: The task completed with an exit code of (0).
"CRF Analysis.job" (CRF Analysis.bat)
        Started 8/28/2014 7:00:00 AM
"CRF Analysis.job" (CRF Analysis.bat)
        Finished 8/28/2014 7:00:09 AM
        Result: The task completed with an exit code of (0).
[ ***** Most recent entry is above this line ***** ]

Corona688 · August 29, 2014, 12:19pm

It fulfilled your specifications and worked on the data you gave. If you wanted dates from the future instead of today's, you could have informed the program of that, or at least informed me of that. Perhaps I was being too "clever" and automating it when all you wanted was something easy to plug values you wanted into?

What you were expecting was -- and remains -- unclear. Tell me exactly what you wish to match for, and exactly what output you expect from it, don't tell me my code is bad for not reading your mind.

That's exactly what I did. The sed chops it into blocks so awk knows where records begin and end, and beyond that, you can do whatever you please.

"grep" doesn't work that way of course -- it matches lines and only lines. If you want something which understands "if line 1 matches something, do something with line 2, print part of line 3, delete part of line 4" that's exactly what awk is for.

I'll keep my example simpler this time, less chance of misunderstanding, and it's easier to improve a working script than figure out what a broken one's supposed to do.

sed 's/^"/\n"/' np_log.txt | awk '$1 ~ FILE { print }' RS="" FS="\n" OFS="\n" ORS="\n" FILE="abcd.job" -

Red is the actual awk program -- in this case, just "print the entire record if the first field matches FILE". I had it as index() before to make it more robust, this is a simpler illustration of what I'm doing.

Green defines variables. I moved them to the back to make it clearer. RS, FS, OFS, ORS are special variables, tailored to your data here so a "record" is exactly one data block and a "field" is exactly one line( so line 1 is $1, line 2 is $2, etc, $NF is the last line, $0 is the entire data block, etc).

Blue is the input file to read. There can be one or more. - reads from standard input.

ajju · August 31, 2014, 11:15pm

corona688:

It fulfilled your specifications and worked on the data you gave. If you wanted dates from the future instead of today's, you could have informed the program of that, or at least informed me of that. Perhaps I was being too "clever" and automating it when all you wanted was something easy to plug values you wanted into?

What you were expecting was -- and remains -- unclear. Tell me exactly what you wish to match for, and exactly what output you expect from it, don't tell me my code is bad for not reading your mind.

That's exactly what I did. The sed chops it into blocks so awk knows where records begin and end, and beyond that, you can do whatever you please.

"grep" doesn't work that way of course -- it matches lines and only lines. If you want something which understands "if line 1 matches something, do something with line 2, print part of line 3, delete part of line 4" that's exactly what awk is for.

I'll keep my example simpler this time, less chance of misunderstanding, and it's easier to improve a working script than figure out what a broken one's supposed to do.
sed 's/^"/\n"/' np_log.txt | awk '$1 ~ FILE { print }' RS="" FS="\n" OFS="\n" ORS="\n" FILE="abcd.job" -
Red is the actual awk program -- in this case, just "print the entire record if the first field matches FILE". I had it as index() before to make it more robust, this is a simpler illustration of what I'm doing.

Green defines variables. I moved them to the back to make it clearer. RS, FS, OFS, ORS are special variables, tailored to your data here so a "record" is exactly one data block and a "field" is exactly one line( so line 1 is $1, line 2 is $2, etc, $NF is the last line, $0 is the entire data block, etc).

Blue is the input file to read. There can be one or more. - reads from standard input.

Thanks, can we add log after the given statement to check that job for the current completed or not...

Corona688 · September 4, 2014, 1:38am

Show input and corresponding output. Relevant input and relevant output. I don't want to be testing the wrong thing again.

Don_Cragun · September 13, 2014, 10:20pm

Riverstone was also using the names ajju and dani1234 to avoid infraction. The latter two have been banned.