A shell script to run a script which don't get terminated and send a pattern from the output by mail

Hi Guys,

I am very new to shell script and I need your help here to write a script. Actually, I have a script abc.sh which don't get terminated itself. So I need to design a script to run this script, save the output to a file, search for a given string in the output and if it exists send those lines over email.

Also, the script abc.sh need to be terminated as it don't get terminated. Now, I am running the script and have to do CTRL+C to get to next prompt.

Any suggestions is highly appreciated.

Thanks,
Sam

Whenever you start a thread in this forum it is a good idea to tell us what operating system and shell you're using.

We could be lazy and say that you should rewrite abc.sh to do what it is supposed to do and exit instead of looping forever, but since we have no idea what it is supposed to do, this might not be possible.

If abc.sh runs forever, it is not possible to write a script that runs it and then checks the contents of a file. One could start it asynchronously and occasionally check its output file, but since you haven't told us what output file(s) it creates nor what output you're hoping to find, it is hard for us to guess at how to do that.

If you showed us the contents of abc.sh (in CODE tags), showed us the output it produces (in CODE tags), and showed us the email you're hoping to produce (in CODE tags); we might have a much better chance of helping you.

Hi Don,

I will explain it in a better way. Linux is the operating system and using bash shell.

So the requirement is as below:

We have a script called testit_tas.sh to test the servers which don't get terminated and we pass CTRL+C to kill it everytime. Below is the content of the script:

!#/bin/bash

./ctftestdump 10.240.56.1 8100 querytas.txt

So when we run this script (./testit_tas.sh) we get this below output on the terminal and it don't get completed. So we kill it manually

[sasahu@cts-pro-mgmt-srv1 idc_test_tool]$ ./testit_tas.sh
./testit_tas.sh: line 1: !#/bin/bash: No such file or directory
Level Zero : trying to connect/close host 10.240.56.1 at port 8100...
in connect
out connect
Send:[5022=LoginUser|5028=red3|5029=red3|5026=1]
Send:[5022=QueryTas|4=564|5=EA|5045=1|5026=1003]
Send:[5022=QueryTas|4=193|5=E:CUSAN.E|5045=1|5026=1004]
Send:[5022=QueryTas|4=558|5=E|5045=1|5026=1005]
Send:[5022=QueryTas|4=627|5=E:VOD|5045=1|5026=1006]
Send:[5022=QueryTas|4=824|5=E:AASB-ME|5045=1|5026=1007]
Send:[5022=QueryTas|4=205|5=E:SHFIN|5045=1|5026=1008]
Send:[5022=QueryTas|4=216|5=E:2INVE|5045=1|5026=1009]
Send:[5022=QueryTas|4=257|5=E:AT00000FACC2-EUR|5045=1|5026=1010]
Send:[5022=QueryTas|4=873|5=E:AHAN|5045=1|5026=1011]
5026=1|5001=0<ETX>
5022=QueryTas|4=205|5=E:SHFIN|5045=1|5026=1008|5001=-46<ETX>
5022=QueryTas|4=824|5=E:AASB-ME|5045=1|5026=1007|5001=-46<ETX>
5022=QueryTas|4=216|5=E:2INVE|5045=1|5026=1009|5001=-46<ETX>
5001=-203|5026=1003<ETX>
5001=-203|5026=1004<ETX>
5001=-203|5026=1005<ETX>
5001=-203|5026=1010<ETX>
5001=-203|5026=1011<ETX>
^C[sasahu@cts-pro-mgmt-srv1 idc_test_tool]$

So from above output, we need to search the lines for 5001=-46<ETX> and capture these lines. -46 is error. So we want to capture this and alert only these lines over email to specific people.

Once this script is written, we will schedule it in crontab which will run 3 times to do this check.

Also, I know that to abort the script we can find the process id and kill it but am not sure how to include this.

TIA.

Regards,
Sambit

On top of what Don Cragun asked, there are some more informations that could help to taylor a solution:

Is ctftestdump your (company's) own command or script, i.e. are you responsible and in a position to analyse / modify it? How is it to be terminated under normal circumstances? Does it deliver a (meaningful?) exit status? Is that reliable and could be exploited for your purpose?

You seemingly post stdout from your command / script - what is querytas.txt for? After how many 5001=-46<ETX> occurrences should the new script terminate / send the e-mails? How long should it wait for that string? Is 46 the only error number? Is <ETX> that literal ASCII string or the ASCII control code "end of text" (0x03)?
cron sends e-mails with a job's output when it finishes - would that be sufficient to accomplish the task?

BTW: The shebang you tried to use is the wrong way round. Make it #! .

Hi Rudy,

Below are some of the responses to your queries:

Is

ctftestdump 

your (company's) own command or script, i.e. are you responsible and in a position to analyse / modify it? How is it to be terminated under normal circumstances? Does it deliver a (meaningful?) exit status? Is that reliable and could be exploited for your purpose?

I think its a binary file which accepts arguments and we cannot modify it. We cant exploit it.

what is

querytas.txt

for?

It holds the queries/requests which need to be sent.

[sasahu@cts-pro-mgmt-srv1 idc_test_tool]$ cat querytas.txt
5022=LoginUser|5028=red3|5029=red3|5026=1
5022=QueryTas|4=564|5=EA|5045=1|5026=1003
5022=QueryTas|4=193|5=E:CUSAN.E|5045=1|5026=1004
5022=QueryTas|4=558|5=E|5045=1|5026=1005
5022=QueryTas|4=627|5=E:VOD|5045=1|5026=1006
5022=QueryTas|4=824|5=E:AASB-ME|5045=1|5026=1007
5022=QueryTas|4=205|5=E:SHFIN|5045=1|5026=1008
5022=QueryTas|4=216|5=E:2INVE|5045=1|5026=1009
5022=QueryTas|4=257|5=E:AT00000FACC2-EUR|5045=1|5026=1010
5022=QueryTas|4=873|5=E:AHAN|5045=1|5026=1011

After how many

5001=-46<ETX>

occurrences should the new script terminate / send the e-mails? How long should it wait for that string? Is 46 the only error number?

When we run the

testit_tas.sh

, it gives the output instantly. There is nothing like we need to wait for some number of occurences of -46. It gives the output instantly and we can terminate it instantly. Below is some information about the output:
So there are three possibilities for the output (remember the response will have the 5026=xxxxx which matches the request)

  1. Data is returned
3=564|4=564|5=AAL|269=15|55=40002379|5061=2989|5055=2045|479=t|10=55.28|11=3|1206=0|16=1519894802.4412|12=52.62|13=10|2001=0|2003=0|478=t|5026=117032<ETX>
5001=0|5026=117032<ETX>

the first line is the data itself, with the 5026=117032
the second line is the response terminator 5001=0

  1. Error -46 is returned
5022=QueryTas|4=824|5=E:AASB-ME|5045=1|5026=27023|5001=-46<ETX>

This is what we want to capture and send an alert over email.

  1. A different error is returned.
5001=-203|5026=87029<ETX>

In this case the error is -203. There could be other errors. This is not urgent issue and we don't need to capture it for now.

cron sends e-mails with a job's output when it finishes - would that be sufficient to accomplish the task?

It would be better if we can just report the lines(represents which markets) contains

5001=-46

(Error) so that further investigation can be done.

So, in short we need to run the

testit_tas.sh

3 times a day to check the server and if there are error (-46) in the output, we will report specific user over email. I hope, I was able to define the requirement

Thanks,
Sam

Hmm - it's not quite clear how the command terminates, or why it does not. Nor, if the number of error lines is predefined or open. Howsoever, how far would

MAILTO=userlist
0 23 * * *    /absolute/path/to/ctftestdump 10.240.56.1 8100 querytas.txt | /absolute/path/to/grep "5001=-46<ETX>"

in crontab get you? It extracts the required error from the output and mails it to userlist. Adapt the date / time fields according to your needs.

Hi Rudy,

This we can do but the problem is how we can terminate. I have tried few things.

Below is the output when I am just running the script.

[sasahu@cts-pro-mgmt-srv1 idc_test_tool]$ /opt/operations/vbenghiat/idc_test_tool/ctftestdump 10.240.56.1 8100 querytas.txt
Level Zero : trying to connect/close host 10.240.56.1 at port 8100...
in connect
out connect
Send:[5022=LoginUser|5028=red3|5029=red3|5026=1]
Send:[5022=QueryTas|4=564|5=EA|5045=1|5026=1003]
Send:[5022=QueryTas|4=193|5=E:CUSAN.E|5045=1|5026=1004]
Send:[5022=QueryTas|4=558|5=E|5045=1|5026=1005]
Send:[5022=QueryTas|4=627|5=E:VOD|5045=1|5026=1006]
Send:[5022=QueryTas|4=824|5=E:AASB-ME|5045=1|5026=1007]
Send:[5022=QueryTas|4=205|5=E:SHFIN|5045=1|5026=1008]
Send:[5022=QueryTas|4=216|5=E:2INVE|5045=1|5026=1009]
Send:[5022=QueryTas|4=257|5=E:AT00000FACC2-EUR|5045=1|5026=1010]
Send:[5022=QueryTas|4=873|5=E:AHAN|5045=1|5026=1011]
5026=1|5001=0<ETX>
5022=QueryTas|4=205|5=E:SHFIN|5045=1|5026=1008|5001=-46<ETX>
3=193|4=193|5=E:CUSAN.E|269=15|10=2.76|11=15250|55=161047660|192=10786|211=6|25=161047|668=1|16=1520860247.6936|12=2.75|13=17773|5026=1004<ETX>
5001=0|5026=1004<ETX>
3=564|4=564|5=EA|269=15|55=91222753|5061=179204|5055=2046|478=t|12=127.12|13=1|16=1520860342.8007|10=129.35|11=9|2001=0|2003=0|479=p|5026=1003<ETX>
5001=0|5026=1003<ETX>
3=873|4=873|5=E:AHAN|269=15|10=5.49|12=5|13=1500|25=95909000|16=1520845149.9286|11=999|5026=1011<ETX>
5001=0|5026=1011<ETX>
3=558|4=558|5=E|269=15|55=91554105528000|5061=250530|5055=2169|479=t|11=7|16=1520860554.1262|12=33.75|13=18|10=33.79|2001=15|2003=0|478=p|5026=1005<ETX>
5001=0|5026=1005<ETX>
5022=QueryTas|4=824|5=E:AASB-ME|5045=1|5026=1007|5001=-46<ETX>
5022=QueryTas|4=216|5=E:2INVE|5045=1|5026=1009|5001=-46<ETX>
3=257|4=257|5=E:AT00000FACC2-EUR|269=15|1642=0|1643=0|298=0|285=0|277=0|297=0|284=0|276=0|442=0|460=0|16=1520828196.5021|5026=1010<ETX>
5001=0|5026=1010<ETX>
^C
[sasahu@cts-pro-mgmt-srv1 idc_test_tool]$

Below are the errors:

[sasahu@cts-pro-mgmt-srv1 idc_test_tool]$ /opt/operations/vbenghiat/idc_test_tool/ctftestdump 10.240.56.1 8100 querytas.txt > tmp.txt
Send:[5022=LoginUser|5028=red3|5029=red3|5026=1]
Send:[5022=QueryTas|4=564|5=EA|5045=1|5026=1003]
Send:[5022=QueryTas|4=193|5=E:CUSAN.E|5045=1|5026=1004]
Send:[5022=QueryTas|4=558|5=E|5045=1|5026=1005]
Send:[5022=QueryTas|4=627|5=E:VOD|5045=1|5026=1006]
Send:[5022=QueryTas|4=824|5=E:AASB-ME|5045=1|5026=1007]
Send:[5022=QueryTas|4=205|5=E:SHFIN|5045=1|5026=1008]
Send:[5022=QueryTas|4=216|5=E:2INVE|5045=1|5026=1009]
Send:[5022=QueryTas|4=257|5=E:AT00000FACC2-EUR|5045=1|5026=1010]
Send:[5022=QueryTas|4=873|5=E:AHAN|5045=1|5026=1011]

So as per your thought, we can do this in cron:

[sasahu@cts-pro-mgmt-srv1 idc_test_tool]$ /opt/operations/vbenghiat/idc_test_tool/ctftestdump 10.240.56.1 8100 querytas.txt 2> /dev/null | grep "5001=-46<ETX>"
5022=QueryTas|4=824|5=E:AASB-ME|5045=1|5026=1007|5001=-46<ETX>
5022=QueryTas|4=205|5=E:SHFIN|5045=1|5026=1008|5001=-46<ETX>
5022=QueryTas|4=216|5=E:2INVE|5045=1|5026=1009|5001=-46<ETX>
^C
[sasahu@cts-pro-mgmt-srv1 idc_test_tool]$

And we get the desired output which we can send it over email. But the issue is how we can terminate. Is it some way we can kill the running PID here. This can be resolved then.

thanks,
Sam

I don't have the slightest idea about how your ctftestdump works and what input it needs. Could an end , exit , or similar command be entered into querytas.txt ?

Try replacing testit_tas.sh with:

#!/bin/bash
IAm=${0##*/}
Tmpfile=/tmp/$IAm.$$

trap 'rm -f "$Tmpfile"' EXIT

./ctftestdump 10.240.56.1 8100 querytas.txt > "$Tmpfile" &
pid=$!
sleep 5
kill $pid
grep -F '5001=-46<TX>' "$Tmpfile"

In addition to normal output from the grep , this might also give you a line similar to the following:

testit_tas.sh: line 11: 20884 Terminated: 15          ./ctftestdump 10.240.56.1 8100 querytas.txt > "$Tmpfile"

which is bash 's way of telling you that cftestdump was killed. If you find this line of output objectionable, you can get rid of it by redirecting stderr when you invoke testit_tas.sh :

./testit_tas.sh 2>/dev/null

but doing this will also hide any other diagnostic messages your script might provide. Do not do this until you know that the script is doing what you want it to do.

1 Like

Hi Don,

This is just perfect. Now i just need to add the mail configuration to the script.

Thanks,
Sambit

Hi Sambit,
In post #7 you said that you could take care of the e-mail using cron as suggested by RudiC in post #6. What else needs to be done?

I still find it difficult to believe that ctftestdump doesn't have a way to exit normally...

1 Like