Log checking script

Shell Programming and Scripting
1

So I have a script that looks through a log for a list of words then sends an email with the results. I would like to expand this to do the same for other logs in the same directory but make it a new email with each log. How would I do that?

LOG="pdb.lg psmseis.lg psyseis.lg"        <----- These are the 3 logs I want it to look through but currently it only looks at the first
# REPORT_NAME is where the information to be sent is written to.
REPORT_NAME='/tmp/pdb.lg_report'
# LOGDIR is the directory containing the log.
LOGDIR="/pdb1"
# LIST is the errors being searched for.

    LIST="
    abnormal
    allow*
    attempt*
    beyond
    cannot
    corrupt*
    damage*
    dead
    died
    disappear*
    drastic
    enough
    error
    exceed*
    fail*
    fatal
    illegal
    impossible
    increase
    insufficient
    invalid
    kill*
    missing
    overflow*
    stget
    stop
    system
    unable
    unexpected*
    unknown
    violation
    warn
    wrong
    "
#------------------------------------------------------------------------------
# PROCESSING SECTION
#------------------------------------------------------------------------------

rm ${REPORT_NAME}

for words in $LIST
do
    RESULTS=`grep -i ${words} ${LOGDIR}/${LOG} | grep -v 'exceeded. Automatically increasing from'`
    if [[ ${RESULTS} > "" ]]
    then
        echo -e "\n----------------------------------------------------------------------------" >> ${REPORT_NAME}
        echo -e "SEARCH PATTERN = ${words}" >> ${REPORT_NAME}
        echo -e "----------------------------------------------------------------------------" >> ${REPORT_NAME}
        grep -i ${words} ${LOGDIR}/${LOG} >> ${REPORT_NAME}
    fi
done
#------------------------------------------------------------------------------
# END OF PROCESSING SECTION
#------------------------------------------------------------------------------

#------------------------------------------------------------------------------
# EMAILING SECTION
#------------------------------------------------------------------------------

MESSAGE_HEADER="Please check these possible errors from the /pdb1/pdb.lg log.\n"
MESSAGE_BODY=`cat ${REPORT_NAME}`
MESSAGE_TAIL="\n\nThis email is from ${0}."
MESSAGE="${MESSAGE_HEADER}${MESSAGE_BODY}${MESSAGE_TAIL}"
SUBJECT="ERROR REPORT from ${LOGDIR}/${LOG}"

for USER in ${USERLIST}
do
    echo -e "${MESSAGE}" | mailx -s "${SUBJECT}" ${USER}
done
2

grep can scan multiple files in one go, so it is fine to present it the expanded LOG variable, but, as the directory is missing for the second and third entries, "files are not found". There are several options to cure that:
a) use an array for LOG , and shell's "parameter expansion / pattern substitution":

LOG=(pdb.lg psmseis.lg psyseis.lg)
echo ${LOG[@]/#/${LOGDIR}/}

b) Use a for loop across the files, but don't forget the directory here as well.
c) cat all three files into grep (which will lose the filename output)

You seem to be a for loop enthusiast. Be aware that there may be good reasons to run one instance of grep for every single keyword in the LIST , but you have to sacrifice quite some performance in order to do so. How about some optimisation? Try adding colour=always to highlight the respective matched keyword:

$ cat << EOF >REPORT_NAME

----------------------------------------------------------------------------
SEARCH PATTERN = ${LIST}
----------------------------------------------------------------------------
EOF
$ echo "$LIST" | grep -f- --colour=always  ${LOG[@]/#/${LOGDIR}/} >>REPORT_NAME

And, mail programs usually take (comma separated?) user lists immediately:

$ USERLIST="user1,user2,user3"$ echo -e "${MESSAGE}" | mailx -s "${SUBJECT}" ${USERLIST}
3

change

LOG="pdb.lg psmseis.lg psyseis.lg"

to

LOG="{pdb.lg,psmseis.lg,psyseis.lg}"

Read up on Brace Expansion in your favorite shell's man page.

There're other issues with the script to debug/fix I'm not going to go into atm.

4

just to clarify, the other logs are in the same directory as they first log but i would like to expand this eventually to be able to use across other db environment but im taking it one step at a time. I do have a userlist in the script i just didnt include it in the posted code for privacy reasons. this is the email that the original script creates if this helps. disclaimer: i didnt write the original script but i have been given the task of upgrading it as a project.

Please check these possible errors from the /pdb1/pdb.lg log.

----------------------------------------------------------------------------
SEARCH PATTERN = abnormal
----------------------------------------------------------------------------

[2020/04/13@07:42:17.885-0500] P-30355      T-140597209511744 I SRV    14: (794)   Usernum 716 terminated abnormally. 
[2020/04/13@10:37:32.364-0500] P-30367      T-139842457388864 I SRV    15: (794)   Usernum 715 terminated abnormally. 
[2020/04/13@11:14:57.815-0500] P-30320      T-140658028422976 I SRV    11: (794)   Usernum 719 terminated abnormally. 
[2020/04/13@14:50:49.182-0500] P-30399      T-140378140444480 I SRV    17: (794)   Usernum 713 terminated abnormally. 
[2020/04/13@17:47:39.479-0500] P-30331      T-139866618869568 I SRV    12: (794)   Usernum 718 terminated abnormally. 
[2020/04/13@17:54:00.450-0500] P-30320      T-140658028422976 I SRV    11: (794)   Usernum 719 terminated abnormally. 
[2020/04/13@17:57:30.993-0500] P-30423      T-140633268692800 I SRV    22: (794)   Usernum 708 terminated abnormally. 
[2020/04/13@19:17:05.347-0500] P-30344      T-140503546509120 I SRV    13: (794)   Usernum 717 terminated abnormally. 
[2020/04/14@01:24:36.615-0500] P-30331      T-139866618869568 I SRV    12: (794)   Usernum 718 terminated abnormally. 
[2020/04/14@01:24:45.980-0500] P-30320      T-140658028422976 I SRV    11: (794)   Usernum 719 terminated abnormally. 
[2020/04/14@13:33:46.572-0500] P-30355      T-140597209511744 I SRV    14: (794)   Usernum 716 terminated abnormally. 
[2020/04/14@15:33:18.741-0500] P-30399      T-140378140444480 I SRV    17: (794)   Usernum 713 terminated abnormally. 
[2020/04/14@18:18:41.118-0500] P-30412      T-140339427333952 I SRV    20: (794)   Usernum 710 terminated abnormally. 
[2020/04/15@01:07:55.407-0500] P-30308      T-140475794827072 I SRV    10: (794)   Usernum 720 terminated abnormally. 
[2020/04/15@01:25:21.829-0500] P-30344      T-140503546509120 I SRV    13: (794)   Usernum 717 terminated abnormally. 
[2020/04/15@01:45:59.626-0500] P-30399      T-140378140444480 I SRV    17: (794)   Usernum 713 terminated abnormally. 
[2020/04/15@01:45:59.660-0500] P-30308      T-140475794827072 I SRV    10: (794)   Usernum 720 terminated abnormally. 
[2020/04/15@07:11:10.400-0500] P-30399      T-140378140444480 I SRV    17: (794)   Usernum 713 terminated abnormally. 
[2020/04/15@07:31:43.706-0500] P-30344      T-140503546509120 I SRV    13: (794)   Usernum 717 terminated abnormally. 
[2020/04/15@10:18:32.844-0500] P-30403      T-140013961852736 I SRV    18: (794)   Usernum 712 terminated abnormally. 
[2020/04/15@15:17:32.587-0500] P-30412      T-140339427333952 I SRV    20: (794)   Usernum 710 terminated abnormally. 
[2020/04/15@15:56:16.336-0500] P-30399      T-140378140444480 I SRV    17: (794)   Usernum 713 terminated abnormally. 
[2020/04/15@18:17:16.439-0500] P-30399      T-140378140444480 I SRV    17: (794)   Usernum 713 terminated abnormally. 
[2020/04/16@01:41:47.610-0500] P-30367      T-139842457388864 I SRV    15: (794)   Usernum 715 terminated abnormally. 
[2020/04/16@01:41:47.613-0500] P-30320      T-140658028422976 I SRV    11: (794)   Usernum 719 terminated abnormally.

----------------------------------------------------------------------------
SEARCH PATTERN = allow*
----------------------------------------------------------------------------

[2020/04/12@13:25:51.351-0500] P-28111      T-139668866561856 I BROKER  0: (12813) Allowed index cursors (-c): 2904 
[2020/04/12@13:25:51.439-0500] P-28111      T-139668866561856 I BROKER  0: (10836) Database connections are not allowed at this time.

----------------------------------------------------------------------------
SEARCH PATTERN = attempt*
----------------------------------------------------------------------------

[2020/04/13@17:54:50.850-0500] P-30408      T-140270759073600 I SRV    19: (1280)  Connection timed out on socket=255 for usernum 711, attempt disconnect. 
[2020/04/13@17:55:37.442-0500] P-30403      T-140013961852736 I SRV    18: (1280)  Connection timed out on socket=255 for usernum 712, attempt disconnect. 
[2020/04/13@18:07:13.250-0500] P-30412      T-140339427333952 I SRV    20: (1280)  Connection timed out on socket=255 for usernum 710, attempt disconnect. 
[2020/04/14@00:25:52.738-0500] P-30308      T-140475794827072 I SRV    10: (1280)  Connection timed out on socket=255 for usernum 720, attempt disconnect. 
[2020/04/14@17:53:54.530-0500] P-30378      T-139793563330368 I SRV    16: (1280)  Connection timed out on socket=255 for usernum 714, attempt disconnect. 
[2020/04/15@03:37:59.906-0500] P-30331      T-139866618869568 I SRV    12: (1280)  Connection timed out on socket=255 for usernum 718, attempt disconnect. 
[2020/04/15@12:17:05.890-0500] P-30308      T-140475794827072 I SRV    10: (1280)  Connection timed out on socket=255 for usernum 720, attempt disconnect.

----------------------------------------------------------------------------
SEARCH PATTERN = error
----------------------------------------------------------------------------

[2020/04/13@07:42:17.886-0500] P-30355      T-140597209511744 I SRV    14: (-----) TCP/IP write error occurred with errno 32
[2020/04/13@10:37:32.365-0500] P-30367      T-139842457388864 I SRV    15: (-----) TCP/IP write error occurred with errno 32
[2020/04/13@11:14:57.817-0500] P-30320      T-140658028422976 I SRV    11: (-----) TCP/IP write error occurred with errno 32
[2020/04/13@14:50:49.183-0500] P-30399      T-140378140444480 I SRV    17: (-----) TCP/IP write error occurred with errno 32
[2020/04/13@17:47:39.481-0500] P-30331      T-139866618869568 I SRV    12: (-----) TCP/IP write error occurred with errno 32
[2020/04/13@17:54:00.451-0500] P-30320      T-140658028422976 I SRV    11: (-----) TCP/IP write error occurred with errno 32
[2020/04/13@17:54:50.851-0500] P-30408      T-140270759073600 I SRV    19: (-----) TCP/IP write error occurred with errno 32
[2020/04/13@17:55:37.444-0500] P-30403      T-140013961852736 I SRV    18: (-----) TCP/IP write error occurred with errno 32
[2020/04/13@17:57:30.995-0500] P-30423      T-140633268692800 I SRV    22: (-----) TCP/IP write error occurred with errno 32
[2020/04/13@18:07:13.252-0500] P-30412      T-140339427333952 I SRV    20: (-----) TCP/IP write error occurred with errno 32
[2020/04/13@19:17:05.348-0500] P-30344      T-140503546509120 I SRV    13: (-----) TCP/IP write error occurred with errno 32
[2020/04/14@00:25:52.740-0500] P-30308      T-140475794827072 I SRV    10: (-----) TCP/IP write error occurred with errno 32
[2020/04/14@01:24:36.617-0500] P-30331      T-139866618869568 I SRV    12: (-----) TCP/IP write error occurred with errno 32
[2020/04/14@01:24:45.982-0500] P-30320      T-140658028422976 I SRV    11: (-----) TCP/IP write error occurred with errno 32
[2020/04/14@13:33:46.573-0500] P-30355      T-140597209511744 I SRV    14: (-----) TCP/IP write error occurred with errno 32
[2020/04/14@15:33:18.742-0500] P-30399      T-140378140444480 I SRV    17: (-----) TCP/IP write error occurred with errno 32
[2020/04/14@17:53:54.536-0500] P-30378      T-139793563330368 I SRV    16: (-----) TCP/IP write error occurred with errno 32
[2020/04/14@18:18:41.190-0500] P-30412      T-140339427333952 I SRV    20: (-----) TCP/IP write error occurred with errno 32
[2020/04/15@01:07:55.408-0500] P-30308      T-140475794827072 I SRV    10: (-----) TCP/IP write error occurred with errno 32
[2020/04/15@01:25:21.831-0500] P-30344      T-140503546509120 I SRV    13: (-----) TCP/IP write error occurred with errno 32
[2020/04/15@01:45:59.629-0500] P-30399      T-140378140444480 I SRV    17: (-----) TCP/IP write error occurred with errno 32
[2020/04/15@01:45:59.661-0500] P-30308      T-140475794827072 I SRV    10: (-----) TCP/IP write error occurred with errno 32
[2020/04/15@03:37:59.908-0500] P-30331      T-139866618869568 I SRV    12: (-----) TCP/IP write error occurred with errno 32
[2020/04/15@07:11:10.402-0500] P-30399      T-140378140444480 I SRV    17: (-----) TCP/IP write error occurred with errno 32
[2020/04/15@07:31:43.707-0500] P-30344      T-140503546509120 I SRV    13: (-----) TCP/IP write error occurred with errno 32
[2020/04/15@10:18:32.846-0500] P-30403      T-140013961852736 I SRV    18: (-----) TCP/IP write error occurred with errno 32
[2020/04/15@12:17:05.891-0500] P-30308      T-140475794827072 I SRV    10: (-----) TCP/IP write error occurred with errno 32
[2020/04/15@15:17:32.588-0500] P-30412      T-140339427333952 I SRV    20: (-----) TCP/IP write error occurred with errno 32
[2020/04/15@15:56:16.337-0500] P-30399      T-140378140444480 I SRV    17: (-----) TCP/IP write error occurred with errno 32
[2020/04/15@18:17:16.441-0500] P-30399      T-140378140444480 I SRV    17: (-----) TCP/IP write error occurred with errno 32
[2020/04/16@01:41:47.611-0500] P-30367      T-139842457388864 I SRV    15: (-----) TCP/IP write error occurred with errno 32
[2020/04/16@01:41:47.614-0500] P-30320      T-140658028422976 I SRV    11: (-----) TCP/IP write error occurred with errno 32

----------------------------------------------------------------------------
SEARCH PATTERN = kill*
----------------------------------------------------------------------------

[2020/04/15@15:30:21.000+0000] P-9021       T-140038969972608 I ABL   435: (298)   KILL signal received.
5

thumbelina,
you'll have to debug your script (as you said) "taking it one step at a time" given the suggestions provided previously by RudyC and the others.
The best approach is to do the debugging iteratively given your own environment specifics and asking specific questions along the way.
Try it out and see how far it gets you given the previous suggestions.
Don't forget to use code tags when posting code and/or data samples.

6

I am struggling with the C part. can you explain a little and where to put this in at.

7

I'm afraid I wasn't too clear. The cat I gave was NOT part of the c) option. It is meant to replace the multiple echo es starting the REPORT_NAME file.