# Log checking script **URL:** \ **Category:** Shell Programming and Scripting\ **Created:** [April 17, 2020, 11:08am UTC](https://community.unix.com/t/log-checking-script/377986 "2020-04-17T11:08:21Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![thumbelina](https://community.unix.com/user_avatar/community.unix.com/thumbelina/32/13930_2.png) [@thumbelina](https://community.unix.com/u/thumbelina)\ **Post date:** [April 17, 2020, 11:08am UTC](https://community.unix.com/t/log-checking-script/377986/1 "2020-04-17T11:08:21Z")
So I have a script that looks through a log for a list of words then sends an email with the results. I would like to expand this to do the same for other logs in the same directory but make it a new email with each log. How would I do that? ```nohighlight LOG="pdb.lg psmseis.lg psyseis.lg" <----- These are the 3 logs I want it to look through but currently it only looks at the first # REPORT_NAME is where the information to be sent is written to. REPORT_NAME='/tmp/pdb.lg_report' # LOGDIR is the directory containing the log. LOGDIR="/pdb1" # LIST is the errors being searched for. LIST=" abnormal allow* attempt* beyond cannot corrupt* damage* dead died disappear* drastic enough error exceed* fail* fatal illegal impossible increase insufficient invalid kill* missing overflow* stget stop system unable unexpected* unknown violation warn wrong " #------------------------------------------------------------------------------ # PROCESSING SECTION #------------------------------------------------------------------------------ rm ${REPORT_NAME} for words in $LIST do RESULTS=`grep -i ${words} ${LOGDIR}/${LOG} | grep -v 'exceeded. Automatically increasing from'` if [[${RESULTS} > ""]] then echo -e "\n----------------------------------------------------------------------------" >> ${REPORT_NAME} echo -e "SEARCH PATTERN = ${words}" >> ${REPORT_NAME} echo -e "----------------------------------------------------------------------------" >> ${REPORT_NAME} grep -i ${words} ${LOGDIR}/${LOG} >> ${REPORT_NAME} fi done #------------------------------------------------------------------------------ # END OF PROCESSING SECTION #------------------------------------------------------------------------------ #------------------------------------------------------------------------------ # EMAILING SECTION #------------------------------------------------------------------------------ MESSAGE_HEADER="Please check these possible errors from the /pdb1/pdb.lg log.\n" MESSAGE_BODY=`cat ${REPORT_NAME}` MESSAGE_TAIL="\n\nThis email is from ${0}." MESSAGE="${MESSAGE_HEADER}${MESSAGE_BODY}${MESSAGE_TAIL}" SUBJECT="ERROR REPORT from ${LOGDIR}/${LOG}" for USER in ${USERLIST} do echo -e "${MESSAGE}" | mailx -s "${SUBJECT}" ${USER} done ``` ---
**Author:** ![RudiC](https://community.unix.com/letter_avatar/rudic/32/6_449c533f586939b6e5f246632a46421f.png) [@RudiC](https://community.unix.com/u/RudiC)\ **Post date:** [April 17, 2020, 1:08pm UTC](https://community.unix.com/t/log-checking-script/377986/2 "2020-04-17T13:08:03Z")
`grep` can scan multiple files in one go, so it is fine to present it the expanded `LOG` variable, but, as the directory is missing for the second and third entries, "files are not found". There are several options to cure that: a) use an array for `LOG` , and shell's "parameter expansion / pattern substitution": ```nohighlight LOG=(pdb.lg psmseis.lg psyseis.lg) echo ${LOG[@]/#/${LOGDIR}/} ``` b) Use a `for` loop across the files, but don't forget the directory here as well. c) `cat` all three files into `grep` (which will lose the filename output) You seem to be a `for` loop enthusiast. Be aware that there may be good reasons to run one instance of `grep` for every single keyword in the `LIST` , but you have to sacrifice quite some performance in order to do so. How about some optimisation? Try adding `colour=always` to highlight the respective matched keyword: ```nohighlight $ cat << EOF >REPORT_NAME ---------------------------------------------------------------------------- SEARCH PATTERN = ${LIST} ---------------------------------------------------------------------------- EOF $ echo "$LIST" | grep -f- --colour=always ${LOG[@]/#/${LOGDIR}/} >>REPORT_NAME ``` And, mail programs usually take (comma separated?) user lists immediately: ```nohighlight $ USERLIST="user1,user2,user3"$ echo -e "${MESSAGE}" | mailx -s "${SUBJECT}" ${USERLIST} ``` ---
**Author:** ![vgersh99](https://community.unix.com/user_avatar/community.unix.com/vgersh99/32/14851_2.png) [@vgersh99](https://community.unix.com/u/vgersh99)\ **Post date:** [April 17, 2020, 1:12pm UTC](https://community.unix.com/t/log-checking-script/377986/3 "2020-04-17T13:12:52Z")
change ```nohighlight LOG="pdb.lg psmseis.lg psyseis.lg" ``` to ```nohighlight LOG="{pdb.lg,psmseis.lg,psyseis.lg}" ``` Read up on `Brace Expansion` in your favorite shell's `man` page. There're other issues with the script to debug/fix I'm not going to go into atm. ---
**Author:** ![thumbelina](https://community.unix.com/user_avatar/community.unix.com/thumbelina/32/13930_2.png) [@thumbelina](https://community.unix.com/u/thumbelina)\ **Post date:** [April 17, 2020, 3:37pm UTC](https://community.unix.com/t/log-checking-script/377986/4 "2020-04-17T15:37:42Z")
just to clarify, the other logs are in the same directory as they first log but i would like to expand this eventually to be able to use across other db environment but im taking it one step at a time. I do have a userlist in the script i just didnt include it in the posted code for privacy reasons. this is the email that the original script creates if this helps. disclaimer: i didnt write the original script but i have been given the task of upgrading it as a project. Please check these possible errors from the /pdb1/pdb.lg log. ---------------------------------------------------------------------------- SEARCH PATTERN = abnormal ---------------------------------------------------------------------------- ```nohighlight [2020/04/13@07:42:17.885-0500] P-30355 T-140597209511744 I SRV 14: (794) Usernum 716 terminated abnormally. [2020/04/13@10:37:32.364-0500] P-30367 T-139842457388864 I SRV 15: (794) Usernum 715 terminated abnormally. [2020/04/13@11:14:57.815-0500] P-30320 T-140658028422976 I SRV 11: (794) Usernum 719 terminated abnormally. [2020/04/13@14:50:49.182-0500] P-30399 T-140378140444480 I SRV 17: (794) Usernum 713 terminated abnormally. [2020/04/13@17:47:39.479-0500] P-30331 T-139866618869568 I SRV 12: (794) Usernum 718 terminated abnormally. [2020/04/13@17:54:00.450-0500] P-30320 T-140658028422976 I SRV 11: (794) Usernum 719 terminated abnormally. [2020/04/13@17:57:30.993-0500] P-30423 T-140633268692800 I SRV 22: (794) Usernum 708 terminated abnormally. [2020/04/13@19:17:05.347-0500] P-30344 T-140503546509120 I SRV 13: (794) Usernum 717 terminated abnormally. [2020/04/14@01:24:36.615-0500] P-30331 T-139866618869568 I SRV 12: (794) Usernum 718 terminated abnormally. [2020/04/14@01:24:45.980-0500] P-30320 T-140658028422976 I SRV 11: (794) Usernum 719 terminated abnormally. [2020/04/14@13:33:46.572-0500] P-30355 T-140597209511744 I SRV 14: (794) Usernum 716 terminated abnormally. [2020/04/14@15:33:18.741-0500] P-30399 T-140378140444480 I SRV 17: (794) Usernum 713 terminated abnormally. [2020/04/14@18:18:41.118-0500] P-30412 T-140339427333952 I SRV 20: (794) Usernum 710 terminated abnormally. [2020/04/15@01:07:55.407-0500] P-30308 T-140475794827072 I SRV 10: (794) Usernum 720 terminated abnormally. [2020/04/15@01:25:21.829-0500] P-30344 T-140503546509120 I SRV 13: (794) Usernum 717 terminated abnormally. [2020/04/15@01:45:59.626-0500] P-30399 T-140378140444480 I SRV 17: (794) Usernum 713 terminated abnormally. [2020/04/15@01:45:59.660-0500] P-30308 T-140475794827072 I SRV 10: (794) Usernum 720 terminated abnormally. [2020/04/15@07:11:10.400-0500] P-30399 T-140378140444480 I SRV 17: (794) Usernum 713 terminated abnormally. [2020/04/15@07:31:43.706-0500] P-30344 T-140503546509120 I SRV 13: (794) Usernum 717 terminated abnormally. [2020/04/15@10:18:32.844-0500] P-30403 T-140013961852736 I SRV 18: (794) Usernum 712 terminated abnormally. [2020/04/15@15:17:32.587-0500] P-30412 T-140339427333952 I SRV 20: (794) Usernum 710 terminated abnormally. [2020/04/15@15:56:16.336-0500] P-30399 T-140378140444480 I SRV 17: (794) Usernum 713 terminated abnormally. [2020/04/15@18:17:16.439-0500] P-30399 T-140378140444480 I SRV 17: (794) Usernum 713 terminated abnormally. [2020/04/16@01:41:47.610-0500] P-30367 T-139842457388864 I SRV 15: (794) Usernum 715 terminated abnormally. [2020/04/16@01:41:47.613-0500] P-30320 T-140658028422976 I SRV 11: (794) Usernum 719 terminated abnormally. ``` ---------------------------------------------------------------------------- SEARCH PATTERN = allow\* ---------------------------------------------------------------------------- ```nohighlight [2020/04/12@13:25:51.351-0500] P-28111 T-139668866561856 I BROKER 0: (12813) Allowed index cursors (-c): 2904 [2020/04/12@13:25:51.439-0500] P-28111 T-139668866561856 I BROKER 0: (10836) Database connections are not allowed at this time. ``` ---------------------------------------------------------------------------- SEARCH PATTERN = attempt\* ---------------------------------------------------------------------------- ```nohighlight [2020/04/13@17:54:50.850-0500] P-30408 T-140270759073600 I SRV 19: (1280) Connection timed out on socket=255 for usernum 711, attempt disconnect. [2020/04/13@17:55:37.442-0500] P-30403 T-140013961852736 I SRV 18: (1280) Connection timed out on socket=255 for usernum 712, attempt disconnect. [2020/04/13@18:07:13.250-0500] P-30412 T-140339427333952 I SRV 20: (1280) Connection timed out on socket=255 for usernum 710, attempt disconnect. [2020/04/14@00:25:52.738-0500] P-30308 T-140475794827072 I SRV 10: (1280) Connection timed out on socket=255 for usernum 720, attempt disconnect. [2020/04/14@17:53:54.530-0500] P-30378 T-139793563330368 I SRV 16: (1280) Connection timed out on socket=255 for usernum 714, attempt disconnect. [2020/04/15@03:37:59.906-0500] P-30331 T-139866618869568 I SRV 12: (1280) Connection timed out on socket=255 for usernum 718, attempt disconnect. [2020/04/15@12:17:05.890-0500] P-30308 T-140475794827072 I SRV 10: (1280) Connection timed out on socket=255 for usernum 720, attempt disconnect. ``` ---------------------------------------------------------------------------- SEARCH PATTERN = error ---------------------------------------------------------------------------- ```nohighlight [2020/04/13@07:42:17.886-0500] P-30355 T-140597209511744 I SRV 14: (-----) TCP/IP write error occurred with errno 32 [2020/04/13@10:37:32.365-0500] P-30367 T-139842457388864 I SRV 15: (-----) TCP/IP write error occurred with errno 32 [2020/04/13@11:14:57.817-0500] P-30320 T-140658028422976 I SRV 11: (-----) TCP/IP write error occurred with errno 32 [2020/04/13@14:50:49.183-0500] P-30399 T-140378140444480 I SRV 17: (-----) TCP/IP write error occurred with errno 32 [2020/04/13@17:47:39.481-0500] P-30331 T-139866618869568 I SRV 12: (-----) TCP/IP write error occurred with errno 32 [2020/04/13@17:54:00.451-0500] P-30320 T-140658028422976 I SRV 11: (-----) TCP/IP write error occurred with errno 32 [2020/04/13@17:54:50.851-0500] P-30408 T-140270759073600 I SRV 19: (-----) TCP/IP write error occurred with errno 32 [2020/04/13@17:55:37.444-0500] P-30403 T-140013961852736 I SRV 18: (-----) TCP/IP write error occurred with errno 32 [2020/04/13@17:57:30.995-0500] P-30423 T-140633268692800 I SRV 22: (-----) TCP/IP write error occurred with errno 32 [2020/04/13@18:07:13.252-0500] P-30412 T-140339427333952 I SRV 20: (-----) TCP/IP write error occurred with errno 32 [2020/04/13@19:17:05.348-0500] P-30344 T-140503546509120 I SRV 13: (-----) TCP/IP write error occurred with errno 32 [2020/04/14@00:25:52.740-0500] P-30308 T-140475794827072 I SRV 10: (-----) TCP/IP write error occurred with errno 32 [2020/04/14@01:24:36.617-0500] P-30331 T-139866618869568 I SRV 12: (-----) TCP/IP write error occurred with errno 32 [2020/04/14@01:24:45.982-0500] P-30320 T-140658028422976 I SRV 11: (-----) TCP/IP write error occurred with errno 32 [2020/04/14@13:33:46.573-0500] P-30355 T-140597209511744 I SRV 14: (-----) TCP/IP write error occurred with errno 32 [2020/04/14@15:33:18.742-0500] P-30399 T-140378140444480 I SRV 17: (-----) TCP/IP write error occurred with errno 32 [2020/04/14@17:53:54.536-0500] P-30378 T-139793563330368 I SRV 16: (-----) TCP/IP write error occurred with errno 32 [2020/04/14@18:18:41.190-0500] P-30412 T-140339427333952 I SRV 20: (-----) TCP/IP write error occurred with errno 32 [2020/04/15@01:07:55.408-0500] P-30308 T-140475794827072 I SRV 10: (-----) TCP/IP write error occurred with errno 32 [2020/04/15@01:25:21.831-0500] P-30344 T-140503546509120 I SRV 13: (-----) TCP/IP write error occurred with errno 32 [2020/04/15@01:45:59.629-0500] P-30399 T-140378140444480 I SRV 17: (-----) TCP/IP write error occurred with errno 32 [2020/04/15@01:45:59.661-0500] P-30308 T-140475794827072 I SRV 10: (-----) TCP/IP write error occurred with errno 32 [2020/04/15@03:37:59.908-0500] P-30331 T-139866618869568 I SRV 12: (-----) TCP/IP write error occurred with errno 32 [2020/04/15@07:11:10.402-0500] P-30399 T-140378140444480 I SRV 17: (-----) TCP/IP write error occurred with errno 32 [2020/04/15@07:31:43.707-0500] P-30344 T-140503546509120 I SRV 13: (-----) TCP/IP write error occurred with errno 32 [2020/04/15@10:18:32.846-0500] P-30403 T-140013961852736 I SRV 18: (-----) TCP/IP write error occurred with errno 32 [2020/04/15@12:17:05.891-0500] P-30308 T-140475794827072 I SRV 10: (-----) TCP/IP write error occurred with errno 32 [2020/04/15@15:17:32.588-0500] P-30412 T-140339427333952 I SRV 20: (-----) TCP/IP write error occurred with errno 32 [2020/04/15@15:56:16.337-0500] P-30399 T-140378140444480 I SRV 17: (-----) TCP/IP write error occurred with errno 32 [2020/04/15@18:17:16.441-0500] P-30399 T-140378140444480 I SRV 17: (-----) TCP/IP write error occurred with errno 32 [2020/04/16@01:41:47.611-0500] P-30367 T-139842457388864 I SRV 15: (-----) TCP/IP write error occurred with errno 32 [2020/04/16@01:41:47.614-0500] P-30320 T-140658028422976 I SRV 11: (-----) TCP/IP write error occurred with errno 32 ``` ---------------------------------------------------------------------------- SEARCH PATTERN = kill\* ---------------------------------------------------------------------------- ```nohighlight [2020/04/15@15:30:21.000+0000] P-9021 T-140038969972608 I ABL 435: (298) KILL signal received. ``` ---
**Author:** ![vgersh99](https://community.unix.com/user_avatar/community.unix.com/vgersh99/32/14851_2.png) [@vgersh99](https://community.unix.com/u/vgersh99)\ **Post date:** [April 17, 2020, 3:57pm UTC](https://community.unix.com/t/log-checking-script/377986/5 "2020-04-17T15:57:16Z")
thumbelina, you'll have to debug your script (as you said) "taking it one step at a time" given the suggestions provided previously by RudyC and the others. The best approach is to do the debugging iteratively given your own environment specifics and asking specific questions along the way. Try it out and see how far it gets you given the previous suggestions. Don't forget to use code tags when posting code and/or data samples. ---
**Author:** ![thumbelina](https://community.unix.com/user_avatar/community.unix.com/thumbelina/32/13930_2.png) [@thumbelina](https://community.unix.com/u/thumbelina)\ **Post date:** [April 17, 2020, 4:48pm UTC](https://community.unix.com/t/log-checking-script/377986/6 "2020-04-17T16:48:59Z")
> [@rudic](#): > > `grep` can scan multiple files in one go, so it is fine to present it the expanded `LOG` variable, but, as the directory is missing for the second and third entries, "files are not found". There are several options to cure that: > a) use an array for `LOG` , and shell's "parameter expansion / pattern substitution": > > ```plaintext > LOG=(pdb.lg psmseis.lg psyseis.lg) > echo ${LOG[@]/#/${LOGDIR}/} > > ``` > > b) Use a `for` loop across the files, but don't forget the directory here as well. > c) `cat` all three files into `grep` (which will lose the filename output) > > You seem to be a `for` loop enthusiast. Be aware that there may be good reasons to run one instance of `grep` for every single keyword in the `LIST` , but you have to sacrifice quite some performance in order to do so. How about some optimisation? Try adding `colour=always` to highlight the respective matched keyword: > > ```plaintext > $ cat << EOF >REPORT_NAME > > ---------------------------------------------------------------------------- > SEARCH PATTERN = ${LIST} > ---------------------------------------------------------------------------- > EOF > $ echo "$LIST" | grep -f- --colour=always ${LOG[@]/#/${LOGDIR}/} >>REPORT_NAME > > ``` > > And, mail programs usually take (comma separated?) user lists immediately: > > ```plaintext > $ USERLIST="user1,user2,user3"$ echo -e "${MESSAGE}" | mailx -s "${SUBJECT}" ${USERLIST} > > ``` I am struggling with the C part. can you explain a little and where to put this in at. ---
**Author:** ![RudiC](https://community.unix.com/letter_avatar/rudic/32/6_449c533f586939b6e5f246632a46421f.png) [@RudiC](https://community.unix.com/u/RudiC)\ **Post date:** [April 18, 2020, 6:34am UTC](https://community.unix.com/t/log-checking-script/377986/7 "2020-04-18T06:34:39Z")
I'm afraid I wasn't too clear. The `cat` I gave was NOT part of the c) option. It is meant to replace the multiple `echo` es starting the REPORT\_NAME file. ---
**Author:** ![system](https://community.unix.com/uploads/default/original/2X/c/c7e9e16a595259edafe88615719d5c5dcda035c2.png) [@system](https://community.unix.com/u/system)\ **Post date:** [September 25, 2020, 1:11am UTC](https://community.unix.com/t/log-checking-script/377986/8 "2020-09-25T01:11:09Z")
Automatically bumped on September 25, 2020, 1:11am UTC ---
**Author:** ![Neo](https://community.unix.com/user_avatar/community.unix.com/neo/32/22838_2.png) [@Neo](https://community.unix.com/u/Neo)\ **Post date:** [September 25, 2020, 3:02am UTC](https://community.unix.com/t/log-checking-script/377986/9 "2020-09-25T03:02:01Z")
Closed on September 25, 2020, 3:02am UTC