OS : Linux 2.6.9-67 - Red Hat Enterprise Linux ES release 4
Looking for a script that reads the following log files that gets generated everynight between 2 - 5am
Master_App_20090717.log
Master_App1_20090717.log
Master_App2_20090717.log
Master_App3_20090717.log
Master_App4_20090717.log
Master_App5_20090717.log
One exception:
The log file Master_App_20090717.log gets data written into every 30 minutes in the 24hr period. This data is appended into the file and remains there. But each time the extract and transload start it start with the current time stamp in it begin line and end with current time stamp for a successful end or it could be Failed end.
In any case those entries remain in the file until we fix the issue and the next load starts and the results of next load will append to the same log file for that same day and will starts with the same begin line. So we need to make sure that script should not report previous failure that happend 35 mins ago and was reported by the same script 30 minutes ago. So we need to place a check that if it does find a failed end and it is more then 35 minutes ago then ignore it and continue reading the file and then report accordingly.
We will run this script in crontab
One script will run every morning after 5:00 am once all the load is finished or we will run 5 script seperately after a specific time to check the status
The other script will run 5 minutes pass every 30 minutes.
*************************************************************************************************
Success Message:
2009-02-25 07:24:24]Master_App start_time=[20090213 000000] end_time=[20090213 002959] ... begin
[Extract-2009-02-25 07:24:24]Execute Extract_App ... Begin
[Extract-2009-02-25 07:24:24]Execute ex_nhv_104052_s ...
[Extract-2009-02-25 07:26:14]Execute ex_nhv_104052_s ... Success
[Extract-2009-02-25 07:26:14]Execute ex_nhv_104114_s ...
[Extract-2009-02-25 07:28:06]Execute ex_nhv_104114_s ... Success
[Extract-2009-02-25 07:28:07]Execute nhv_104094_s ...
[Extract-2009-02-25 07:29:58]Execute nhv_104094_s ... Success
[Extract-2009-02-25 07:29:58]Execute Extract_App ... Success
[Transload-2009-02-25 07:29:59]Execute Transload_App ... Begin
[Transload-2009-02-25 07:29:59]Execute tl_nhv_ua_merge ...
[Transload-2009-02-25 07:30:14]Execute tl_nhv_ua_merge ... Success
[Transload-2009-02-25 07:30:16]Execute tl_nhv_104094_s ...
[Transload-2009-02-25 07:39:56]Execute tl_nhv_104094_s ... Success
[Transload-2009-02-25 07:39:57]Execute Transload_APP ... Success
Archiving data files ... Success
[2009-02-25 07:39:58]Master_App start_time=[20090213 000000] end_time=[20090213 002959] [Successful end]!
*************************************************************************************************
Failure Message:
[2009-02-25 08:55:55]Master_App start_time=[20090213 003000] end_time=[20090213 005959] ... begin
[Extract-2009-02-25 08:55:55]Execute Extract_App ... Begin
[Extract-2009-02-25 08:55:55]Execute ex_nhv_104052_s ...
[Extract-2009-02-25 08:57:43]Execute ex_nhv_104052_s ... Success
[Extract-2009-02-25 08:57:43]Execute ex_nhv_104114_s ...
[Extract-2009-02-25 08:59:33]Execute ex_nhv_104114_s ... Success
[Extract-2009-02-25 08:59:33]Execute nhv_104094_s ...
[Extract-2009-02-25 09:01:34]Execute nhv_104094_s ... Success
[Extract-2009-02-25 09:01:34]Execute Extract_App ... Success
[Transload-2009-02-25 09:01:35]Execute Transload_eHealth_h ... Begin
[Transload-2009-02-25 09:01:35]Execute tl_nhv_ua_merge ...
[Transload-2009-02-25 09:01:51]Execute tl_nhv_ua_merge ... Success
[Transload-2009-02-25 09:01:52]Execute tl_nhv_104094_s ... ERROR: ORA-03114: not connected to ORACLE DECLARE Archive_Files.prg bak exp_prdmmp_dev_20090123.dmp.gz......
....................................................ERROR at line 1: ORA-03113: end-of-file on communication channel SP2-0670: Internal number conversion failed Usage: { EXIT | QUIT } [ SUCCESS | FAILURE | WARNING | n | <variable> | :<bindvariable> ] [ COMMIT | ROLLBACK ]
[Transload-2009-02-25 09:33:19]Execute tl_nhv_104094_s ... Fail
[Transload-2009-02-25 09:33:20]Execute Transload_App ... Fail
[2009-02-25 09:33:20]Master_App start_time=[20090213 003000] end_time=[20090213 005959] [Failed end]!
*************************************************************************************************
Output file :
We could have 2 output files one for script that runs after 5 am and one for every 30 minute script
or
we could have 5 output files one for each log plus one for 30 minutes script output log so altogether 6 outfiles.
In the output file
First Delete the content of the file if any then
Leave blank if it is successful
Write "ETL Load Failed" if it finds any failed end.
---------- Post updated at 10:33 AM ---------- Previous update was at 10:32 AM ----------
Thank you in advance for your Help as i am desperately seeking a solution. Thank you and all the best