I create a shell script program (AIX OS) that flags any logs that certain keywords such as memsize, sasfoundation, and real time. My program isn't working maybe because my syntax logistic isn't correct.
Here are examples of the logs:
file1.log.02897 file2.log.02896 filez.log.02899 file4.log.02898 file5.log.028999 file6.log.02898 filex.log.028999 filem.log.028999
file11.log.028999 filexx.log.028999 file56.log.028999 filexz.log.028999 fileq.log.028999
first three attachments are examples of data in each the log files:
last attachment progflag.csv is an example of how the result needs to be displayed in the file.
Here is my code:
#!/bin/bash
#tar current logs and move the tar package to /tmp/log . Untar a logs
sudo cd /home/user/myfiles ; tar cf - */*/*.log.* | ( cd /tmp/log ; tar xf - )
#change all files under log directory to write and execute
chmod 775 files =/tmp/log/*.*
#set target threshold to 3600 sec. (1hour)
echo threshold = 3600
#read in multiple .log files from directory /tmp/log
for file in /tmp/log/*.log.*; do
while read -r LINE; do
echo "$LINE"
# search for keywords line by line
flag_line=$(echo $LINE | awk '{print $NF}')
# if MEMSIZE has a value, extract value and then Output column name MEMSIZE and values to progflag.csv
if echo $flag_line "MEMSIZE" | cut -d: -f2 then
echo MEMSIZE=$flag_line
# extract numeric value from row titled REAL TIME
if echo $flag_line "REAL TIME:" | cut -d: -f3 then
echo SEC=$flag_line
# if sec is greater than threshold then output column name SECOND and values to progflag.csv
IF $sec > $thresold then
echo SECOND=sec
# if path SASFoundation exist than extract SASFoundation and add SASFoundation to alias SASEXE. Output column name SASEXE and
value to progflag.csv
if echo $flag_line '/SASFoundation' grep SASFoundation
echo SASEXE=$flag_line
#output each title of filename with extracted data to progflag.csv
Echo filename=$LINE
Echo $MEMSIZE $SECOND $SASEXE $LINE
done < "$file" >> progflag.csv
#send the attachment via email
if progflag.csv then
(cat ./progflag.csv)|mailx -s "subject text" -a "Programs flagged" receiver@domain.com
#after process is completed, remove all files from directory tmp/log
sudo rm -rf $files $progflag.csv
I'm trying to figure out what some of your code is trying to do. Can you please answer the following questions:
Will the user running this script have read access to files matching the pattern /home/user/myfiles/*/*/*.log.* ? If yes, why are you wasting time making copies of these files? If not, why are you copying these files into a directory and changing their access permissions such that every user with access to your system will be able to read this (presumably private) data?
Why does anyone need write or execute permission to the log files you are copying while this script is running?
What exactly are you trying to do with the command?:
chmod 775 files =/tmp/log/*.*
What it does do is change the permissions of a file named files and of all files in the directory located in the current working directory named =/tmp/log whose names in that directory contain the string .log. so that any process running with the user ID of the file's owner or with a group ID matching the file's group will have read, write, and execute permissions and all other processes will be able to read and execute them.
Will the user running this script have read access to files matching the pattern /home/user/myfiles/*/*/*.log.* ?
no. I will schedule the shell script program to run using ascheduler.
sudo cd /home/user/myfiles ; tar cf - */*/*.log | ( cd /tmp/log ; tar xf - )
This script copies log files from one directory to another. The log files are generated with the current dates on the end of the .log extension for example file1.log.05242016
I ran your code which is awesome. I had one issue. I checked the brackets in the script and the code appears to be good. This is the error message that I am getting
awk: 0602-542 there is an extra ] character.
The source line is 12.
The error context is
v=$2;gsub(/^[\\/ ]*/,"",v);gsub(/ *$/,"",v);K[L[$1]]=v}
awk: 0602-521 There is a regular expresssion error.
[] imbalance
0 progflag.csv
When the code ran and populated the progflag.csv file,
columns MEMSIZE, SECOND and PATH were blank. The column for filename
was populated with the names of each log files from the directory (tmp/log)
a. What I want the program to do is to only read in all files that have an extension
.log.
a. there are other files in the log directory that doesn't have .log. extension
1 a. Also , What I want the program to output in the proflag.csv for example
if the following information in file12.log doesn't exist, such as MEMSIZE ,
a directory path will the text /SASFoundation and the variable for Real Time value
(Real Time 0.1) is less than 1.0, there is no output for
file12.log in progflag.csv .
b. Another example, if in file9.log, the following variables and values exist for
example "MEMSIZE=200" and variable Real Time \(Real Time 4.0\) row value is
More than 1.0, output the values and the column names to the proflag.csv.
c. last example, if in file25.log, there is a directory path will text /SASFoundation
and the variable Real Time \(Real Time 0.2\) row value is less than 1.0 output the
values and its column name to the proflag.csv
Here is an example(1a-c) of the result: progflag.csv
MEMSIZE SECOND PATH filename:
200 4.0 file1.log
SASFoundation file25.log
last, step only send an email if progflag.csv file rows are populated.
So, how do you add an if then condition statement to awk in your code
program?
for example
awk
if MEMSIZE then Or if SECOND ( SECOND > 1.0)
is greater than 1.0 (SECOND is the alias for Real Time. In this case Real Time 4.0)
or SASFoundation (Path is the alias for SASFoundation ) then output to
progflag.csv