Automate Job monitoring

cg_saran · November 13, 2012, 7:17pm

Hi,
I have a project which i need to automate some manual work. We have a 100 jobs in Linux and Unix environment. We do job monitoring daily. that is taking more that 2 to 3 hrs for us because we use autorep -j <jobname> command for each and every job. That takes more time for us to do.
We have to automate it.
We have a list of jobs in xcel/csv files. We have added this command in all jobs like below.
autorep -j job_one
autorep -j job_two
.
.
.
autorep -j job_one_hundred
If i am running this command i am getting below output.
bash-2.05$ autorep -j gxxxxxxx
Job Name Last Start Last End ST Run Pri/Xit
____________________________ ____________________ ____________________ __ _______ ___
gxxxxxxx(box job) 11/08/2012 16:00:08 11/08/2012 16:03:43 SU 18472621/1
gyyyyyy(command job) 11/08/2012 16:00:49 11/08/2012 16:01:01 SU 18472621/1
Now my requirement, i have to create shell script. It should read all autorep command from xcel/csv sheet, Run autorep command and output should store in xcel/csv, send that mail to specific group. Output Excel should have Job Name(command and box jobs), Last Start, Last End and ST of all jobs and should display.
How could i achive this?

Yoda · November 13, 2012, 7:44pm

You can write a while loop to read all autorep command, execute, write status to a file:-

print "Job Name, Last Start, Last End, ST" > job_monitor.csv
while read autorep_cmd
do
   $autorep_cmd | awk 'NR>2 { print $1, $2 " " $3, $4 " " $5, $6 } ' OFS="," >> job_monitor.csv
done < job_cmd.csv

cat job_monitor.csv | mailx -s "Job Monitor" someone@domain.com

cg_saran · November 13, 2012, 8:10pm

I am new to unix.
Could you please give me exact code for this?

Yoda · November 13, 2012, 8:21pm

The code which I provided will work, just replace file name: job_cmd.csv with the CSV file name that you have with the list of autorep commands.

cg_saran · November 13, 2012, 8:44pm

i am getting blank file as output.

#!/bin/sh
 
print "Job Name, Last Start, Last End, ST" > job_monitor.csv
while read autorep_cmd
do
   $autorep_cmd | awk 'NR>2 { print $1, $2 " " $3, $4 " " $5, $6 } ' OFS="," >> job_monitor.csv
done < job_cmd.csv

Yoda · November 13, 2012, 9:01pm

What do you have in job_cmd.csv file? Can you please post the output of below two commands:-

head job_cmd.csv

head job_monitor.csv

Please use code tags when posting code segments.

cg_saran · November 13, 2012, 9:10pm

 
$ head job_cmd.csv
autorep -j job_name_xxxxxxx
$ head job_monitor.csv
Job Name, Last Start, Last End, ST $

Yoda · November 13, 2012, 9:14pm

OK, can you run below 2 commands on UNIX prompt and post the results:-

autorep -j job_name_xxxxxxxx

autorep -j job_name_xxxxxxxx | awk 'NR>2 { print $1, $2 " " $3, $4 " " $5, $6 } ' OFS=","

cg_saran · November 13, 2012, 9:17pm

$ autorep -j job_name_xxxxxxxx
Job Name                         Last Start            Last End        ST  Run  Pri/Xit
____________________________ ____________________ ____________________ __ _______ ___
job_name_xxxxxxxxx        11/13/2012  18:00:09 11/13/2012  18:38:13 SU 18546465/1
  job_name_yyyyyyy     11/13/2012  18:00:34 11/13/2012  18:20:36 SU 18546465/1

$ autorep -j job_name_xxxxxxxx | awk 'NR>2 { print $1, $2 " " $3, $4 " " $5, $6 } ' OFS=","
____________________________,____________________ ____________________,__ _______,___
, , ,
job_name_xxxxxxxx,11/13/2012 18:00:09,11/13/2012 18:38:13,SU
job_name_yyyyyyy,11/13/2012 18:00:34,11/13/2012 18:20:36,SU

its a violation for me to give the actual job name.

Yoda · November 13, 2012, 9:20pm

OK, I got it. Put double quotes around the variable and re-try:-

"$autorep_cmd" | awk 'NR>2 { print $1, $2 " " $3, $4 " " $5, $6 } ' OFS="," >> job_monitor.csv

cg_saran · November 13, 2012, 9:27pm

#!/bin/sh
printf "Job Name, Last Start, Last End, ST" > job_monitor.csv
while read autorep_cmd
do
   "$autorep_cmd" | awk 'NR>4 { printf $1, $2 " " $3, $4 " " $5, $6 } ' OFS="," >> job_monitor.csv
done < job_cmd.csv

 
: command not found autorep -j job_name_xxxxxx

Yoda · November 13, 2012, 9:29pm

Dollar sign should be inside double quotes not outside:-

"$autorep_cmd"

cg_saran · November 13, 2012, 9:31pm

: command not found autorep -j job_name

Getting different error.

Yoda · November 13, 2012, 9:49pm

Try:-

eval "$autorep_cmd" | awk 'NR>4 { printf $1, $2 " " $3, $4 " " $5, $6 } ' OFS="," >> job_monitor.csv

cg_saran · November 13, 2012, 9:54pm

No error but blank output file .

 autorep -j 1111111_XXXXX | awk 'NR>4 { print $1, $2 " " $3, $4 " " $5, $6 } ' OFS="," >> job_monitor.csv

This one is working fine

Yoda · November 13, 2012, 10:00pm

That is weird! You can set debug and see what exactly is happening:-

#/bin/sh -xv

cg_saran · November 13, 2012, 10:05pm

Could you please delete the job name in one of your reply? Actual name of the job should NOT appear in the public forum.
Please delete ur reply which has a actual job name.

---------- Post updated at 10:05 PM ---------- Previous update was at 10:03 PM ----------

printf "Job Name, Last Start, Last End, ST" > job_monitor.csv
+ printf 'Job Name, Last Start, Last End, ST'
while read autorep_cmd
do
  eval "$autorep_cmd" | awk 'NR>4 { printf $1, $2 " " $3, $4 " " $5, $6 } ' OFS="," >> job_monitor.csv
done < job_cmd.csv
+ read autorep_cmd
' eval 'autorep -j Job_name
autorep -j Job_name
++ autorep -j $'Job_name\r'
+ awk 'NR>4 { printf $1, $2 " " $3, $4 " " $5, $6 } ' OFS=,
+ read autorep_cmd
' eval 'autorep -j Job_name
autorep -j Job_name
++ autorep -j $'Job_name\r'
+ awk 'NR>4 { printf $1, $2 " " $3, $4 " " $5, $6 } ' OFS=,
+ read autorep_cmd